github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/223#issuecomment-762540514 | https://api.github.com/repos/simonw/sqlite-utils/issues/223 | 762540514 | MDEyOklzc3VlQ29tbWVudDc2MjU0MDUxNA== | 9599 | 2021-01-19T01:14:58Z | 2021-01-19T01:15:54Z | OWNER | Example from https://docs.python.org/3/library/csv.html#csv.reader ```pycon >>> import csv >>> with open('eggs.csv', newline='') as csvfile: ... spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|') ... for row in spamreader: ... print(', '.join(row)) Spam, Spam, Spam, Spam, Spam, Baked Beans Spam, Lovely Spam, Wonderful Spam ``` I'm going to add `--quotechar` as well as `--delimiter`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
788527932 | |
https://github.com/simonw/datasette/issues/1194#issuecomment-762390568 | https://api.github.com/repos/simonw/datasette/issues/1194 | 762390568 | MDEyOklzc3VlQ29tbWVudDc2MjM5MDU2OA== | 9599 | 2021-01-18T17:43:03Z | 2021-01-18T17:43:03Z | OWNER | Should I just blanket copy over any query string argument that starts with an underscore? Any reason _not_ to do that? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
788447787 | |
https://github.com/simonw/datasette/issues/1194#issuecomment-762390401 | https://api.github.com/repos/simonw/datasette/issues/1194 | 762390401 | MDEyOklzc3VlQ29tbWVudDc2MjM5MDQwMQ== | 9599 | 2021-01-18T17:42:38Z | 2021-01-18T17:42:38Z | OWNER | Relevant code: https://github.com/simonw/datasette/blob/a882d679626438ba0d809944f06f239bcba8ee96/datasette/views/table.py#L815-L827 It looks like there are other arguments that may not be persisted too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
788447787 | |
https://github.com/simonw/datasette/issues/1036#issuecomment-762387875 | https://api.github.com/repos/simonw/datasette/issues/1036 | 762387875 | MDEyOklzc3VlQ29tbWVudDc2MjM4Nzg3NQ== | 9599 | 2021-01-18T17:36:36Z | 2021-01-18T17:36:36Z | OWNER | As you can see, I'm pretty paranoid about serving content with `Content-Type` HTTP headers because I'm so worried about execution vulnerabilities. I'm much more comfortable exploring that kind of thing in plugins, since that way people can opt-in to riskier features. You found `datasette-media` which is my most comprehensive exploration of that idea so far - but there's definitely lots of room for more plugins along those lines. Maybe even an output plugin? `.jpg` as an export format which returns the `BLOB` column for a row as a JPEG image with the correct `content-type` header (but first verifies that the binary content does indeed look like a real JPEG) could be interesting. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
725996507 | |
https://github.com/simonw/datasette/issues/1191#issuecomment-761705076 | https://api.github.com/repos/simonw/datasette/issues/1191 | 761705076 | MDEyOklzc3VlQ29tbWVudDc2MTcwNTA3Ng== | 9599 | 2021-01-17T00:35:13Z | 2021-01-17T00:37:51Z | OWNER | I'm going to try using Jinja macros to implement this: https://jinja.palletsprojects.com/en/2.11.x/templates/#macros Maybe using one of these tricks to auto-load the macro? http://codyaray.com/2015/05/auto-load-jinja2-macros | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
787098345 | |
https://github.com/simonw/datasette/issues/1181#issuecomment-761703555 | https://api.github.com/repos/simonw/datasette/issues/1181 | 761703555 | MDEyOklzc3VlQ29tbWVudDc2MTcwMzU1NQ== | 9599 | 2021-01-17T00:24:20Z | 2021-01-17T00:24:40Z | OWNER | Here's the incomplete sketch of a test - to go at the bottom of `test_cli.py`. ```python @pytest.mark.parametrize( "filename", ["test-database (1).sqlite", "database (1).sqlite"] ) def test_weird_database_names(ensure_eventloop, tmpdir, filename): # https://github.com/simonw/datasette/issues/1181 runner = CliRunner() db_path = str(tmpdir / filename) sqlite3.connect(db_path).execute("vacuum") result1 = runner.invoke(cli, [db_path, "--get", "/"]) assert result1.exit_code == 0, result1.output homepage_html = result1.output assert False ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
781262510 | |
https://github.com/simonw/datasette/issues/1191#issuecomment-761703368 | https://api.github.com/repos/simonw/datasette/issues/1191 | 761703368 | MDEyOklzc3VlQ29tbWVudDc2MTcwMzM2OA== | 9599 | 2021-01-17T00:22:46Z | 2021-01-17T00:22:46Z | OWNER | I'm going to prototype this in a branch. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
787098345 | |
https://github.com/simonw/datasette/issues/1191#issuecomment-761703232 | https://api.github.com/repos/simonw/datasette/issues/1191 | 761703232 | MDEyOklzc3VlQ29tbWVudDc2MTcwMzIzMg== | 9599 | 2021-01-17T00:21:31Z | 2021-01-17T00:21:54Z | OWNER | I think this ends up being a whole collection of new plugin hooks, something like: - `include_table_top` - `include_table_bottom` - `include_row_top` - `include_row_bottom` - `include_database_top` - `include_database_bottom` - `include_query_bottom` - `include_query_bottom` - `include_index_bottom` - `include_index_bottom` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
787098345 | |
https://github.com/simonw/datasette/issues/1191#issuecomment-761703022 | https://api.github.com/repos/simonw/datasette/issues/1191 | 761703022 | MDEyOklzc3VlQ29tbWVudDc2MTcwMzAyMg== | 9599 | 2021-01-17T00:20:00Z | 2021-01-17T00:20:00Z | OWNER | Plugins that want to provide extra context to the template can already do so using the `extra_template_vars()` plugin hook. So this hook could work by returning a list of template filenames to be included. Those templates can be bundled with the plugin. Since they are included they will have access to the template context and to any `extra_template_vars()` values. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
787098345 | |
https://github.com/simonw/datasette/issues/657#issuecomment-761179229 | https://api.github.com/repos/simonw/datasette/issues/657 | 761179229 | MDEyOklzc3VlQ29tbWVudDc2MTE3OTIyOQ== | 9599 | 2021-01-15T20:24:35Z | 2021-01-15T20:24:35Z | OWNER | I'm not sure how I missed this issue but it's almost a year later and I'm finally taking a look at your Parquet work. This is yet more evidence that allowing plugins to provide their own custom `Database` objects would be a good idea. I started exploring what Datasette would like on PostgreSQL in #670 - my concern was that I would need to add a large amount of database abstraction code which would dramatically increase the complexity of the core project, but my thinking now is that it might be tractable - Datasette doesn't actually construct SQL in complex ways anywhere outside of the `TableView` class so abstracting away just that bit should be feasible. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
548591089 | |
https://github.com/simonw/datasette/issues/1191#issuecomment-761103910 | https://api.github.com/repos/simonw/datasette/issues/1191 | 761103910 | MDEyOklzc3VlQ29tbWVudDc2MTEwMzkxMA== | 9599 | 2021-01-15T18:19:29Z | 2021-01-15T18:19:29Z | OWNER | This relates to #987 (documented HTML hooks for JavaScript plugins) but is not quite the same thing. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
787098345 | |
https://github.com/simonw/datasette/issues/657#issuecomment-761101878 | https://api.github.com/repos/simonw/datasette/issues/657 | 761101878 | MDEyOklzc3VlQ29tbWVudDc2MTEwMTg3OA== | 9599 | 2021-01-15T18:16:01Z | 2021-01-15T18:16:01Z | OWNER | I think the `startup()` plugin hook at https://docs.datasette.io/en/stable/plugin_hooks.html#startup-datasette should be able to fit this. You can write a plugin which uses that hook to execute `CREATE VIRTUAL TABLE` against one or more databases when Datasette first starts running. Would that work here? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
548591089 | |
https://github.com/simonw/datasette/issues/1187#issuecomment-759875239 | https://api.github.com/repos/simonw/datasette/issues/1187 | 759875239 | MDEyOklzc3VlQ29tbWVudDc1OTg3NTIzOQ== | 9599 | 2021-01-14T02:02:24Z | 2021-01-14T02:02:31Z | OWNER | This plugin hook currently returns a string of JavaScript. It could optionally return this instead of a string: ```json { "script": "string of JavaScript goes here", "module": true } ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
785588942 | |
https://github.com/simonw/datasette/issues/1186#issuecomment-759874332 | https://api.github.com/repos/simonw/datasette/issues/1186 | 759874332 | MDEyOklzc3VlQ29tbWVudDc1OTg3NDMzMg== | 9599 | 2021-01-14T01:59:35Z | 2021-01-14T01:59:35Z | OWNER | Updated documentation: https://docs.datasette.io/en/latest/custom_templates.html#custom-css-and-javascript and https://docs.datasette.io/en/latest/plugin_hooks.html#extra-js-urls-template-database-table-columns-view-name-request-datasette | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
785573793 | |
https://github.com/simonw/sqlite-utils/issues/220#issuecomment-759098964 | https://api.github.com/repos/simonw/sqlite-utils/issues/220 | 759098964 | MDEyOklzc3VlQ29tbWVudDc1OTA5ODk2NA== | 9599 | 2021-01-12T23:19:55Z | 2021-01-12T23:19:55Z | OWNER | I don't think it makes sense to call `.enable_fts()` on a view does it? When I'm working with views and FTS I tend to write my queries against a FTS table for one of the tables that is used by the view - https://docs.datasette.io/en/stable/full_text_search.html#configuring-full-text-search-for-a-table-or-view describes how I do that in Datasette for example. Can you expand on your use-case for FTS and views? I'm ready to be convinced otherwise, but I don't see how it would work right now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
783778672 | |
https://github.com/simonw/datasette/issues/1185#issuecomment-759069342 | https://api.github.com/repos/simonw/datasette/issues/1185 | 759069342 | MDEyOklzc3VlQ29tbWVudDc1OTA2OTM0Mg== | 9599 | 2021-01-12T22:13:18Z | 2021-01-12T22:13:18Z | OWNER | I'm going to change the error message to list the allowed pragmas. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
784628163 | |
https://github.com/simonw/datasette/issues/1185#issuecomment-759067427 | https://api.github.com/repos/simonw/datasette/issues/1185 | 759067427 | MDEyOklzc3VlQ29tbWVudDc1OTA2NzQyNw== | 9599 | 2021-01-12T22:09:21Z | 2021-01-12T22:09:21Z | OWNER | That allow-list was added in #761 but is not currently documented. It's here in the code: https://github.com/simonw/datasette/blob/8e8fc5cee5c78da8334495c6d6257d5612c40792/datasette/utils/__init__.py#L173-L186 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
784628163 | |
https://github.com/simonw/datasette/issues/1185#issuecomment-759066777 | https://api.github.com/repos/simonw/datasette/issues/1185 | 759066777 | MDEyOklzc3VlQ29tbWVudDc1OTA2Njc3Nw== | 9599 | 2021-01-12T22:07:58Z | 2021-01-12T22:07:58Z | OWNER | https://docs.datasette.io/en/stable/sql_queries.html?highlight=pragma#named-parameters documentation is out-of-date as well: > Datasette disallows custom SQL containing the string PRAGMA, as SQLite pragma statements can be used to change database settings at runtime. If you need to include the string "pragma" in a query you can do so safely using a named parameter. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
784628163 | |
https://github.com/simonw/datasette/issues/1183#issuecomment-758356640 | https://api.github.com/repos/simonw/datasette/issues/1183 | 758356640 | MDEyOklzc3VlQ29tbWVudDc1ODM1NjY0MA== | 9599 | 2021-01-12T02:42:08Z | 2021-01-12T02:42:08Z | OWNER | Should Datasette have subcommands for this? `datasette enable-counts data.db` and `datasette disable-counts data.db` and `datasette reset-counts data.db` commands? Maybe. The `sqlite-utils` CLI tool could be used here instead, but that won't be easily available if Datasette was installed as a standalone binary or using `brew install datasette` or `pipx install datasette`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
782708469 | |
https://github.com/simonw/datasette/issues/1183#issuecomment-758356097 | https://api.github.com/repos/simonw/datasette/issues/1183 | 758356097 | MDEyOklzc3VlQ29tbWVudDc1ODM1NjA5Nw== | 9599 | 2021-01-12T02:40:30Z | 2021-01-12T02:40:30Z | OWNER | So how would this work? I think I'm going to automatically use these values if the `_counts` table exists, unless a `ignore_counts_table` boolean setting has been set. I won't bother looking to see if the triggers have been created. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
782708469 | |
https://github.com/simonw/datasette/issues/1091#issuecomment-758283074 | https://api.github.com/repos/simonw/datasette/issues/1091 | 758283074 | MDEyOklzc3VlQ29tbWVudDc1ODI4MzA3NA== | 9599 | 2021-01-11T23:12:46Z | 2021-01-11T23:12:46Z | OWNER | Fantastic! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
742011049 | |
https://github.com/simonw/datasette/issues/1182#issuecomment-757375858 | https://api.github.com/repos/simonw/datasette/issues/1182 | 757375858 | MDEyOklzc3VlQ29tbWVudDc1NzM3NTg1OA== | 9599 | 2021-01-09T22:18:47Z | 2021-01-09T22:18:47Z | OWNER | https://docs.datasette.io/en/latest/ecosystem.html | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
782692159 | |
https://github.com/simonw/datasette/issues/1182#issuecomment-757373741 | https://api.github.com/repos/simonw/datasette/issues/1182 | 757373741 | MDEyOklzc3VlQ29tbWVudDc1NzM3Mzc0MQ== | 9599 | 2021-01-09T22:01:41Z | 2021-01-09T22:01:41Z | OWNER | It can talk about Dogsheep too. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
782692159 | |
https://github.com/simonw/datasette/issues/1182#issuecomment-757373082 | https://api.github.com/repos/simonw/datasette/issues/1182 | 757373082 | MDEyOklzc3VlQ29tbWVudDc1NzM3MzA4Mg== | 9599 | 2021-01-09T21:55:33Z | 2021-01-09T21:55:33Z | OWNER | I'll leave the page there but change it into more of a blurb about the existence of the plugins and tools directories. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
782692159 | |
https://github.com/simonw/datasette/issues/1181#issuecomment-756487966 | https://api.github.com/repos/simonw/datasette/issues/1181 | 756487966 | MDEyOklzc3VlQ29tbWVudDc1NjQ4Nzk2Ng== | 9599 | 2021-01-08T01:25:42Z | 2021-01-08T01:25:42Z | OWNER | I'm going to add a unit test that tries a variety of weird database names. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
781262510 | |
https://github.com/simonw/datasette/issues/1181#issuecomment-756482163 | https://api.github.com/repos/simonw/datasette/issues/1181 | 756482163 | MDEyOklzc3VlQ29tbWVudDc1NjQ4MjE2Mw== | 9599 | 2021-01-08T01:06:23Z | 2021-01-08T01:06:54Z | OWNER | Yes, that logic is definitely at fault. It looks like it applies `urllib.parse.unquote_plus()` AFTER it's tried to do the `-` hash splitting thing, which is why it's failing here: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/views/base.py#L184-L198 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
781262510 | |
https://github.com/simonw/datasette/issues/1091#issuecomment-756453945 | https://api.github.com/repos/simonw/datasette/issues/1091 | 756453945 | MDEyOklzc3VlQ29tbWVudDc1NjQ1Mzk0NQ== | 9599 | 2021-01-07T23:42:50Z | 2021-01-07T23:42:50Z | OWNER | @henry501 it looks like you spotted a bug in the documentation - I just addressed that, the fix is now live here: https://docs.datasette.io/en/latest/deploying.html#running-datasette-behind-a-proxy | { "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
742011049 | |
https://github.com/simonw/datasette/issues/1091#issuecomment-756453010 | https://api.github.com/repos/simonw/datasette/issues/1091 | 756453010 | MDEyOklzc3VlQ29tbWVudDc1NjQ1MzAxMA== | 9599 | 2021-01-07T23:39:58Z | 2021-01-07T23:40:25Z | OWNER | @tballison I think that's the solution! It looks like you need to use this in your config: `ProxyPass /datasette http://127.0.0.1:8001/datasette` Instead of this: `ProxyPass /datasette http://127.0.0.1:8001/` Give that a go and let me know if it fixes it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
742011049 | |
https://github.com/simonw/datasette/issues/1171#issuecomment-756335394 | https://api.github.com/repos/simonw/datasette/issues/1171 | 756335394 | MDEyOklzc3VlQ29tbWVudDc1NjMzNTM5NA== | 9599 | 2021-01-07T19:35:59Z | 2021-01-07T19:35:59Z | OWNER | I requested a D-U-N-S number as a first step in getting a developer certificate: https://developer.apple.com/support/D-U-N-S/ | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778450486 | |
https://github.com/simonw/datasette/issues/1180#issuecomment-756312213 | https://api.github.com/repos/simonw/datasette/issues/1180 | 756312213 | MDEyOklzc3VlQ29tbWVudDc1NjMxMjIxMw== | 9599 | 2021-01-07T18:56:24Z | 2021-01-07T18:56:24Z | OWNER | The `async_call_with_supported_arguments` version should be able to await any of the lazy arguments that are awaitable - can use `await_me_maybe` for that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780767542 | |
https://github.com/simonw/datasette/issues/1180#issuecomment-755500475 | https://api.github.com/repos/simonw/datasette/issues/1180 | 755500475 | MDEyOklzc3VlQ29tbWVudDc1NTUwMDQ3NQ== | 9599 | 2021-01-06T18:43:41Z | 2021-01-06T18:43:41Z | OWNER | Relevant code: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/utils/__init__.py#L919-L940 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780767542 | |
https://github.com/simonw/datasette/issues/1179#issuecomment-755495387 | https://api.github.com/repos/simonw/datasette/issues/1179 | 755495387 | MDEyOklzc3VlQ29tbWVudDc1NTQ5NTM4Nw== | 9599 | 2021-01-06T18:39:23Z | 2021-01-06T18:39:23Z | OWNER | In that case maybe there are three new arguments: `path`, `full_path` and `url`. I'll also add `request.full_path` for consistency with these: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/utils/asgi.py#L77-L90 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780278550 | |
https://github.com/simonw/datasette/issues/1179#issuecomment-755492945 | https://api.github.com/repos/simonw/datasette/issues/1179 | 755492945 | MDEyOklzc3VlQ29tbWVudDc1NTQ5Mjk0NQ== | 9599 | 2021-01-06T18:37:39Z | 2021-01-06T18:37:39Z | OWNER | I think I'll call this `full_path` for consistency with Django. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780278550 | |
https://github.com/simonw/datasette/issues/1179#issuecomment-755489974 | https://api.github.com/repos/simonw/datasette/issues/1179 | 755489974 | MDEyOklzc3VlQ29tbWVudDc1NTQ4OTk3NA== | 9599 | 2021-01-06T18:35:24Z | 2021-01-06T18:35:24Z | OWNER | Django calls this ` HttpRequest.get_full_path()` - for the path plus the querystring. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780278550 | |
https://github.com/simonw/datasette/issues/1179#issuecomment-755486103 | https://api.github.com/repos/simonw/datasette/issues/1179 | 755486103 | MDEyOklzc3VlQ29tbWVudDc1NTQ4NjEwMw== | 9599 | 2021-01-06T18:32:41Z | 2021-01-06T18:34:11Z | OWNER | This parameter will return the URL path, with querystring arguments, to the HTML version of the page - e.g. `/github/issue_comments` or `/github/issue_comments?_sort_desc=created_at` Open questions: - What should it be called? `path` could be misleading since it also includes the querystring. - Should I provide a `url` or `full_url` version which includes `https://blah.com/...`? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780278550 | |
https://github.com/simonw/datasette/issues/782#issuecomment-755484384 | https://api.github.com/repos/simonw/datasette/issues/782 | 755484384 | MDEyOklzc3VlQ29tbWVudDc1NTQ4NDM4NA== | 9599 | 2021-01-06T18:31:14Z | 2021-01-06T18:31:57Z | OWNER | In building https://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_labels=on I discovered the following patterns for importing data into both Pandas and Observable/d3: ```python import pandas df = pandas.read_json( "https://latest-with-plugins.datasette.io/github/issue_comments.json?_shape=array" ) ``` And: ```javascript d3 = require("d3@5") rows = d3.json( "https://latest-with-plugins.datasette.io/github/issue_comments.json?_shape=array" ) ``` Once again I find myself torn on the best possible default. A list of JSON objects is instantly compatible with both `pandas.read_json()` and `d3.json()` - but it leaves nowhere to put the extra information like pagination and suchlike! Even given this I still think the correct default is an object with `"rows"`, `"total"` and `"next_url"` keys. I should commit to that and implement it - this thought exercise has been running for far too long. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755476820 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755476820 | MDEyOklzc3VlQ29tbWVudDc1NTQ3NjgyMA== | 9599 | 2021-01-06T18:24:47Z | 2021-01-06T18:24:47Z | OWNER | Issue fixed - https://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_labels=on displays the correct schemes now. I can't think of a reason anyone on Cloud Run would ever NOT want the `force_https_urls` option, but just in case I've made it so if you pass `--extra-options --setting force_https_urls off` to `publish cloudrun` your setting will be respected. https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/publish/cloudrun.py#L105-L110 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755468795 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755468795 | MDEyOklzc3VlQ29tbWVudDc1NTQ2ODc5NQ== | 9599 | 2021-01-06T18:14:35Z | 2021-01-06T18:14:35Z | OWNER | Deploying that change now to test it. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755163886 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755163886 | MDEyOklzc3VlQ29tbWVudDc1NTE2Mzg4Ng== | 9599 | 2021-01-06T08:37:51Z | 2021-01-06T08:37:51Z | OWNER | Easiest fix would be for `publish cloudrun` to set `force_https_urls`: `datasette publish now` used to do this: https://github.com/simonw/datasette/blob/07e208cc6d9e901b87552c1be2854c220b3f9b6d/datasette/publish/now.py#L59-L63 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1179#issuecomment-755161574 | https://api.github.com/repos/simonw/datasette/issues/1179 | 755161574 | MDEyOklzc3VlQ29tbWVudDc1NTE2MTU3NA== | 9599 | 2021-01-06T08:32:31Z | 2021-01-06T08:32:31Z | OWNER | An optional `path` argument to https://docs.datasette.io/en/stable/plugin_hooks.html#register-output-renderer-datasette which shows the path WITHOUT the `.Notebook` extension would be useful here. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780278550 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755160187 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755160187 | MDEyOklzc3VlQ29tbWVudDc1NTE2MDE4Nw== | 9599 | 2021-01-06T08:29:35Z | 2021-01-06T08:29:35Z | OWNER | https://latest-with-plugins.datasette.io/-/asgi-scope ``` {'asgi': {'spec_version': '2.1', 'version': '3.0'}, 'client': ('169.254.8.129', 54971), 'headers': [(b'host', b'latest-with-plugins.datasette.io'), (b'user-agent', b'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:84.0) Gecko' b'/20100101 Firefox/84.0'), (b'accept', b'text/html,application/xhtml+xml,application/xml;q=0.9,image/' b'webp,*/*;q=0.8'), (b'accept-language', b'en-US,en;q=0.5'), (b'dnt', b'1'), (b'cookie', b'_ga_LL6M7BK6D4=GS1.1.1609886546.49.1.1609886923.0; _ga=GA1.1' b'.894633707.1607575712'), (b'upgrade-insecure-requests', b'1'), (b'x-client-data', b'CgSL6ZsV'), (b'x-cloud-trace-context', b'e776af843c657d2a3da28a73b726e6fe/14187666787557102189;o=1'), (b'x-forwarded-for', b'148.64.98.14'), (b'x-forwarded-proto', b'https'), (b'forwarded', b'for="148.64.98.14";proto=https'), (b'accept-encoding', b'gzip, deflate, br'), (b'content-length', b'0')], 'http_version': '1.1', 'method': 'GET', 'path': '/-/asgi-scope', 'query_string': b'', 'raw_path': b'/-/asgi-scope', 'root_path': '', 'scheme': 'http', 'server': ('169.254.8.130', 8080), 'type': 'http'} ``` Note the `'scheme': 'http'` but also the `(b'x-forwarded-proto', b'https')`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1176#issuecomment-755159583 | https://api.github.com/repos/simonw/datasette/issues/1176 | 755159583 | MDEyOklzc3VlQ29tbWVudDc1NTE1OTU4Mw== | 9599 | 2021-01-06T08:28:20Z | 2021-01-06T08:28:20Z | OWNER | I used `from datasette.utils import path_with_format` in https://github.com/simonw/datasette-export-notebook/blob/0.1/datasette_export_notebook/__init__.py just now. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
779691739 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755158310 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755158310 | MDEyOklzc3VlQ29tbWVudDc1NTE1ODMxMA== | 9599 | 2021-01-06T08:25:31Z | 2021-01-06T08:25:31Z | OWNER | Moving this to the Datasette repo. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755157732 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755157732 | MDEyOklzc3VlQ29tbWVudDc1NTE1NzczMg== | 9599 | 2021-01-06T08:24:12Z | 2021-01-06T08:24:12Z | OWNER | https://latest-with-plugins.datasette.io/fixtures/sortable.json has the bug too - the `next_url` is `http://` when it should be `https://`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755157281 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755157281 | MDEyOklzc3VlQ29tbWVudDc1NTE1NzI4MQ== | 9599 | 2021-01-06T08:23:14Z | 2021-01-06T08:23:14Z | OWNER | https://latest-with-plugins.datasette.io/-/settings says `"force_https_urls": false` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755157066 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755157066 | MDEyOklzc3VlQ29tbWVudDc1NTE1NzA2Ng== | 9599 | 2021-01-06T08:22:47Z | 2021-01-06T08:22:47Z | OWNER | Weird... https://github.com/simonw/datasette/blob/a882d679626438ba0d809944f06f239bcba8ee96/datasette/app.py#L609-L613 ```python def absolute_url(self, request, path): url = urllib.parse.urljoin(request.url, path) if url.startswith("http://") and self.setting("force_https_urls"): url = "https://" + url[len("http://") :] return url ``` That looks like it should work. Needs more digging. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1178#issuecomment-755156606 | https://api.github.com/repos/simonw/datasette/issues/1178 | 755156606 | MDEyOklzc3VlQ29tbWVudDc1NTE1NjYwNg== | 9599 | 2021-01-06T08:21:49Z | 2021-01-06T08:21:49Z | OWNER | https://github.com/simonw/datasette-export-notebook/blob/aec398eab4f34791d240d7bc47b6eec575b357be/datasette_export_notebook/__init__.py#L18-L23 Maybe this is a bug in `datasette.absolute_url`? Perhaps it doesn't take the scheme into account. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
780267857 | |
https://github.com/simonw/datasette/issues/1101#issuecomment-755134771 | https://api.github.com/repos/simonw/datasette/issues/1101 | 755134771 | MDEyOklzc3VlQ29tbWVudDc1NTEzNDc3MQ== | 9599 | 2021-01-06T07:28:01Z | 2021-01-06T07:28:01Z | OWNER | With this structure it will become possible to stream non-newline-delimited JSON array-of-objects too - the `stream_rows()` method could output `[` first, then each row followed by a comma, then `]` after the very last row. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
749283032 | |
https://github.com/simonw/datasette/issues/1101#issuecomment-755133937 | https://api.github.com/repos/simonw/datasette/issues/1101 | 755133937 | MDEyOklzc3VlQ29tbWVudDc1NTEzMzkzNw== | 9599 | 2021-01-06T07:25:48Z | 2021-01-06T07:26:43Z | OWNER | Idea: instead of returning a dictionary, `register_output_renderer` could return an object. The object could have the following properties: - `.extension` - the extension to use - `.can_render(...)` - says if it can render this - `.can_stream(...)` - says if streaming is supported - `async .stream_rows(rows_iterator, send)` - method that loops through all rows and uses `send` to send them to the response in the correct format I can then deprecate the existing `dict` return type for 1.0. | { "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
749283032 | |
https://github.com/simonw/datasette/issues/1101#issuecomment-755128038 | https://api.github.com/repos/simonw/datasette/issues/1101 | 755128038 | MDEyOklzc3VlQ29tbWVudDc1NTEyODAzOA== | 9599 | 2021-01-06T07:10:22Z | 2021-01-06T07:10:22Z | OWNER | Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas: pandas.read_json("https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on", lines=True) | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
749283032 | |
https://github.com/simonw/datasette/issues/1171#issuecomment-754958998 | https://api.github.com/repos/simonw/datasette/issues/1171 | 754958998 | MDEyOklzc3VlQ29tbWVudDc1NDk1ODk5OA== | 9599 | 2021-01-05T23:16:33Z | 2021-01-05T23:16:33Z | OWNER | That's really useful, thanks @rcoup | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778450486 | |
https://github.com/simonw/datasette/issues/782#issuecomment-754958610 | https://api.github.com/repos/simonw/datasette/issues/782 | 754958610 | MDEyOklzc3VlQ29tbWVudDc1NDk1ODYxMA== | 9599 | 2021-01-05T23:15:24Z | 2021-01-05T23:15:24Z | OWNER | https://latest-with-plugins.datasette.io/fixtures/roadside_attraction_characteristics/1.json-preview returns a 500 error at the moment - a KeyError on 'filtered_table_rows_count'. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
627794879 | |
https://github.com/simonw/datasette/issues/576#issuecomment-754957658 | https://api.github.com/repos/simonw/datasette/issues/576 | 754957658 | MDEyOklzc3VlQ29tbWVudDc1NDk1NzY1OA== | 9599 | 2021-01-05T23:12:50Z | 2021-01-05T23:12:50Z | OWNER | See https://docs.datasette.io/en/stable/internals.html | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
497170355 | |
https://github.com/simonw/datasette/issues/576#issuecomment-754957563 | https://api.github.com/repos/simonw/datasette/issues/576 | 754957563 | MDEyOklzc3VlQ29tbWVudDc1NDk1NzU2Mw== | 9599 | 2021-01-05T23:12:37Z | 2021-01-05T23:12:37Z | OWNER | I'm happy with how this has evolved, so I'm closing the issue. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
497170355 | |
https://github.com/simonw/datasette/issues/1176#issuecomment-754957378 | https://api.github.com/repos/simonw/datasette/issues/1176 | 754957378 | MDEyOklzc3VlQ29tbWVudDc1NDk1NzM3OA== | 9599 | 2021-01-05T23:12:03Z | 2021-01-05T23:12:03Z | OWNER | This needs to be done for Datasette 1.0. At the very least I need to ensure it's clear that `datasette.utils` is not part of the public API unless explicitly marked as such. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
779691739 | |
https://github.com/simonw/datasette/issues/1176#issuecomment-754952146 | https://api.github.com/repos/simonw/datasette/issues/1176 | 754952146 | MDEyOklzc3VlQ29tbWVudDc1NDk1MjE0Ng== | 9599 | 2021-01-05T22:57:26Z | 2021-01-05T22:57:26Z | OWNER | Known public APIs might be worth adding type annotations to as well. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
779691739 | |
https://github.com/simonw/datasette/issues/1176#issuecomment-754952040 | https://api.github.com/repos/simonw/datasette/issues/1176 | 754952040 | MDEyOklzc3VlQ29tbWVudDc1NDk1MjA0MA== | 9599 | 2021-01-05T22:57:09Z | 2021-01-05T22:57:09Z | OWNER | It might be neater to move all of the non-public functions into a separate module - `datasette.utils.internal` perhaps. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
779691739 | |
https://github.com/simonw/datasette/issues/1176#issuecomment-754951786 | https://api.github.com/repos/simonw/datasette/issues/1176 | 754951786 | MDEyOklzc3VlQ29tbWVudDc1NDk1MTc4Ng== | 9599 | 2021-01-05T22:56:27Z | 2021-01-05T22:56:43Z | OWNER | Idea: introduce a `@documented` decorator which marks specific functions as part of the public, documented API. The unit tests can then confirm that anything with that decorator is both documented and tested. ```python @documented def escape_css_string(s): return _css_re.sub( lambda m: "\\" + (f"{ord(m.group()):X}".zfill(6)), s.replace("\r\n", "\n"), ) ``` Or maybe `@public`? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
779691739 | |
https://github.com/simonw/datasette/issues/1175#issuecomment-754696725 | https://api.github.com/repos/simonw/datasette/issues/1175 | 754696725 | MDEyOklzc3VlQ29tbWVudDc1NDY5NjcyNQ== | 9599 | 2021-01-05T15:12:30Z | 2021-01-05T15:12:30Z | OWNER | Some tips here: https://github.com/tiangolo/fastapi/issues/78 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
779156520 | |
https://github.com/simonw/datasette/issues/1173#issuecomment-754463845 | https://api.github.com/repos/simonw/datasette/issues/1173 | 754463845 | MDEyOklzc3VlQ29tbWVudDc1NDQ2Mzg0NQ== | 9599 | 2021-01-05T07:41:43Z | 2021-01-05T07:41:43Z | OWNER | https://github.com/oleksis/pyinstaller-manylinux looks useful, via https://twitter.com/oleksis/status/1346341987876823040 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778682317 | |
https://github.com/simonw/datasette/issues/1171#issuecomment-754296761 | https://api.github.com/repos/simonw/datasette/issues/1171 | 754296761 | MDEyOklzc3VlQ29tbWVudDc1NDI5Njc2MQ== | 9599 | 2021-01-04T23:55:44Z | 2021-01-04T23:55:44Z | OWNER | Bit uncomfortable that it looks like you need to include your Apple ID username and password in the CI configuration to do this. I'll use GitHub Secrets for this but I don't like it - I'll definitely setup a dedicated code signing account that's not my access-to-everything AppleID for this. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778450486 | |
https://github.com/simonw/datasette/issues/1171#issuecomment-754295380 | https://api.github.com/repos/simonw/datasette/issues/1171 | 754295380 | MDEyOklzc3VlQ29tbWVudDc1NDI5NTM4MA== | 9599 | 2021-01-04T23:54:32Z | 2021-01-04T23:54:32Z | OWNER | https://github.com/search?l=YAML&q=gon+json&type=Code reveals some examples of people using `gon` in workflows. These look useful: * https://github.com/coherence/hub-server/blob/3b7e9c7c5bce9e244b14b854f1f89d66f53a5a39/.github/workflows/release_build.yml * https://github.com/simoncozens/pilcrow/blob/5abc145e7fb9577086afe47b48fd730cb8195386/.github/workflows/buildapp.yaml | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778450486 | |
https://github.com/simonw/datasette/issues/1171#issuecomment-754287882 | https://api.github.com/repos/simonw/datasette/issues/1171 | 754287882 | MDEyOklzc3VlQ29tbWVudDc1NDI4Nzg4Mg== | 9599 | 2021-01-04T23:40:10Z | 2021-01-04T23:42:32Z | OWNER | This looks VERY useful: https://github.com/mitchellh/gon - " Sign, notarize, and package macOS CLI tools and applications written in any language. Available as both a CLI and a Go library." And it installs like this: brew install mitchellh/gon/gon | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778450486 | |
https://github.com/simonw/datasette/issues/1171#issuecomment-754286783 | https://api.github.com/repos/simonw/datasette/issues/1171 | 754286783 | MDEyOklzc3VlQ29tbWVudDc1NDI4Njc4Mw== | 9599 | 2021-01-04T23:38:18Z | 2021-01-04T23:38:18Z | OWNER | Oh wow maybe I need to Notarize it too? https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778450486 | |
https://github.com/simonw/datasette/issues/1171#issuecomment-754286618 | https://api.github.com/repos/simonw/datasette/issues/1171 | 754286618 | MDEyOklzc3VlQ29tbWVudDc1NDI4NjYxOA== | 9599 | 2021-01-04T23:37:45Z | 2021-01-04T23:37:45Z | OWNER | https://github.com/actions/virtual-environments/issues/1820#issuecomment-719549887 looks useful - not sure if those notes are for iOS or macOS though. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778450486 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754285795 | https://api.github.com/repos/simonw/datasette/issues/93 | 754285795 | MDEyOklzc3VlQ29tbWVudDc1NDI4NTc5NQ== | 9599 | 2021-01-04T23:35:13Z | 2021-01-04T23:35:13Z | OWNER | Next step is to automate this all! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/1152#issuecomment-754285588 | https://api.github.com/repos/simonw/datasette/issues/1152 | 754285588 | MDEyOklzc3VlQ29tbWVudDc1NDI4NTU4OA== | 9599 | 2021-01-04T23:34:30Z | 2021-01-04T23:34:30Z | OWNER | I think the way to do this is to have a new plugin hook that returns two SQL where clauses: one returning a list of resources that the user should be able to access (the allow-list) and one returning a list of resources they are explicitly forbidden from accessing (the deny-list). Either of these can be blank. Datasette can then combine those into a full SQL query and use it to answer the question "show me a list of resources that the user is allowed to perform action X on". It can also answer the existing question, "is user X allowed to perform action Y on resource Z"? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
770598024 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754233960 | https://api.github.com/repos/simonw/datasette/issues/93 | 754233960 | MDEyOklzc3VlQ29tbWVudDc1NDIzMzk2MA== | 9599 | 2021-01-04T21:35:37Z | 2021-01-04T21:35:37Z | OWNER | I tested it by running a `tmate` session on the GitHub macOS machines, and it worked! ``` Mac-1609795972770:tmp runner$ wget 'https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip' --2021-01-04 21:34:10-- https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip Resolving github.com (github.com)... 140.82.114.4 Connecting to github.com (github.com)|140.82.114.4|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/107914493/74658700-4e90-11eb-8f3b-ee77e6dfad90?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210104T213414Z&X-Amz-Expires=300&X-Amz-Signature=6f3c54211077092553590b33a7c36cd052895c9d4619607ad1df094782f64acf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=107914493&response-content-disposition=attachment%3B%20filename%3Ddatasette-0.53-macos-binary.zip&response-content-type=application%2Foctet-stream [following] --2021-01-04 21:34:14-- https://github-production-release-asset-2e65be.s3.amazonaws.com/107914493/74658700-4e90-11eb-8f3b-ee77e6dfad90?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210104T213414Z&X-Amz-Expires=300&X-Amz-Signature=6f3c54211077092553590b33a7c36cd052895c9d4619607ad1df094782f64acf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=107914493&response-content-disposition=attachment%3B%20filename%3Ddatasette-0.53-macos-binary.zip&response-content-type=application%2Foctet-stream Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.43.164 Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.43.164|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 8297283 (7.9M) [applicat… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754229977 | https://api.github.com/repos/simonw/datasette/issues/93 | 754229977 | MDEyOklzc3VlQ29tbWVudDc1NDIyOTk3Nw== | 9599 | 2021-01-04T21:28:01Z | 2021-01-04T21:28:01Z | OWNER | As an experiment, I put the macOS one in a zip file and attached it to the latest release: ``` mkdir datasette-0.53-macos-binary cp dist/datasette datasette-0.53-macos-binary zip -r datasette-0.53-macos-binary.zip datasette-0.53-macos-binary ``` It's available here: https://github.com/simonw/datasette/releases/tag/0.53 - download URL is https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754227543 | https://api.github.com/repos/simonw/datasette/issues/93 | 754227543 | MDEyOklzc3VlQ29tbWVudDc1NDIyNzU0Mw== | 9599 | 2021-01-04T21:23:13Z | 2021-01-04T21:23:13Z | OWNER | ``` (pyinstaller-venv) root@dogsheep:/tmp/pyinstaller-venv# dist/datasette --get /-/databases.json [{"name": ":memory:", "path": null, "size": 0, "is_mutable": true, "is_memory": true, "hash": null}] (pyinstaller-venv) root@dogsheep:/tmp/pyinstaller-venv# ls -lah dist/datasette -rwxr-xr-x 1 root root 8.9M Jan 4 21:05 dist/datasette ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754219002 | https://api.github.com/repos/simonw/datasette/issues/93 | 754219002 | MDEyOklzc3VlQ29tbWVudDc1NDIxOTAwMg== | 9599 | 2021-01-04T21:06:49Z | 2021-01-04T21:22:27Z | OWNER | Works on Linux/Ubuntu too, except I had to do `export BASE=` on a separate line. I also did this: ``` apt-get install python3 python3-venv python3 -m venv pyinstaller-venv source pyinstaller-venv/bin/activate pip install wheel pip install datasette pyinstaller export DATASETTE_BASE=$(python -c 'import os; print(os.path.dirname(__import__("datasette").__file__))') pyinstaller -F \ --add-data "$DATASETTE_BASE/templates:datasette/templates" \ --add-data "$DATASETTE_BASE/static:datasette/static" \ --hidden-import datasette.publish \ --hidden-import datasette.publish.heroku \ --hidden-import datasette.publish.cloudrun \ --hidden-import datasette.facets \ --hidden-import datasette.sql_functions \ --hidden-import datasette.actor_auth_cookie \ --hidden-import datasette.default_permissions \ --hidden-import datasette.default_magic_parameters \ --hidden-import datasette.blob_renderer \ --hidden-import datasette.default_menu_links \ --hidden-import uvicorn \ --hidden-import uvicorn.logging \ --hidden-import uvicorn.loops \ --hidden-import uvicorn.loops.auto \ --hidden-import uvicorn.protocols \ --hidden-import uvicorn.protocols.http \ --hidden-import uvicorn.protocols.http.auto \ --hidden-import uvicorn.protocols.websockets \ --hidden-import uvicorn.protocols.websockets.auto \ --hidden-import uvicorn.lifespan \ --hidden-import uvicorn.lifespan.on \ $(which datasette) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754218545 | https://api.github.com/repos/simonw/datasette/issues/93 | 754218545 | MDEyOklzc3VlQ29tbWVudDc1NDIxODU0NQ== | 9599 | 2021-01-04T21:05:57Z | 2021-01-04T21:05:57Z | OWNER | That BASE= trick seems to work with `zsh` but not with `bash`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754215392 | https://api.github.com/repos/simonw/datasette/issues/93 | 754215392 | MDEyOklzc3VlQ29tbWVudDc1NDIxNTM5Mg== | 9599 | 2021-01-04T20:59:20Z | 2021-01-04T21:03:14Z | OWNER | Updated `pyinstaller` recipe - lots of hidden imports needed now: ``` pip install wheel pip install datasette pyinstaller BASE=$(python -c 'import os; print(os.path.dirname(__import__("datasette").__file__))') \ pyinstaller -F \ --add-data "$BASE/templates:datasette/templates" \ --add-data "$BASE/static:datasette/static" \ --hidden-import datasette.publish \ --hidden-import datasette.publish.heroku \ --hidden-import datasette.publish.cloudrun \ --hidden-import datasette.facets \ --hidden-import datasette.sql_functions \ --hidden-import datasette.actor_auth_cookie \ --hidden-import datasette.default_permissions \ --hidden-import datasette.default_magic_parameters \ --hidden-import datasette.blob_renderer \ --hidden-import datasette.default_menu_links \ --hidden-import uvicorn \ --hidden-import uvicorn.logging \ --hidden-import uvicorn.loops \ --hidden-import uvicorn.loops.auto \ --hidden-import uvicorn.protocols \ --hidden-import uvicorn.protocols.http \ --hidden-import uvicorn.protocols.http.auto \ --hidden-import uvicorn.protocols.websockets \ --hidden-import uvicorn.protocols.websockets.auto \ --hidden-import uvicorn.lifespan \ --hidden-import uvicorn.lifespan.on \ $(which datasette) ``` | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/93#issuecomment-754215793 | https://api.github.com/repos/simonw/datasette/issues/93 | 754215793 | MDEyOklzc3VlQ29tbWVudDc1NDIxNTc5Mw== | 9599 | 2021-01-04T21:00:14Z | 2021-01-04T21:00:14Z | OWNER | ``` (pyinstaller-datasette) pyinstaller-datasette % file dist/datasette dist/datasette: Mach-O 64-bit executable x86_64 (pyinstaller-datasette) pyinstaller-datasette % ls -lah dist/datasette -rwxr-xr-x 1 simon wheel 8.0M Jan 4 12:58 dist/datasette ``` I'm surprised it's only 8MB! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
273944952 | |
https://github.com/simonw/datasette/issues/668#issuecomment-754194996 | https://api.github.com/repos/simonw/datasette/issues/668 | 754194996 | MDEyOklzc3VlQ29tbWVudDc1NDE5NDk5Ng== | 9599 | 2021-01-04T20:18:39Z | 2021-01-04T20:18:39Z | OWNER | I fixed this in #1115 - you can run `--load-extension=spatialite` now and it will look for the extension in common places. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
563347679 | |
https://github.com/simonw/datasette/issues/436#issuecomment-754193501 | https://api.github.com/repos/simonw/datasette/issues/436 | 754193501 | MDEyOklzc3VlQ29tbWVudDc1NDE5MzUwMQ== | 9599 | 2021-01-04T20:15:41Z | 2021-01-04T20:15:41Z | OWNER | Sadly `publish.datasettes.com` was broken by changes to Zeit, and I don't think I'll be bringing it back. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
435819321 | |
https://github.com/simonw/datasette/issues/371#issuecomment-754192873 | https://api.github.com/repos/simonw/datasette/issues/371 | 754192873 | MDEyOklzc3VlQ29tbWVudDc1NDE5Mjg3Mw== | 9599 | 2021-01-04T20:14:28Z | 2021-01-04T20:14:28Z | OWNER | Now that Digital Ocean has App Platform this is less necessary, especially since the documentation covers how to use App Platform here: https://docs.datasette.io/en/stable/deploying.html#deploying-using-buildpacks | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
377156339 | |
https://github.com/simonw/datasette/issues/102#issuecomment-754192267 | https://api.github.com/repos/simonw/datasette/issues/102 | 754192267 | MDEyOklzc3VlQ29tbWVudDc1NDE5MjI2Nw== | 9599 | 2021-01-04T20:13:19Z | 2021-01-04T20:13:19Z | OWNER | I'm more likely to do Lambda than Elastic Beanstalk, especially now the size limit for Lambdas has been increased as part of their support for Docker. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274264175 | |
https://github.com/simonw/datasette/issues/221#issuecomment-754191699 | https://api.github.com/repos/simonw/datasette/issues/221 | 754191699 | MDEyOklzc3VlQ29tbWVudDc1NDE5MTY5OQ== | 9599 | 2021-01-04T20:12:14Z | 2021-01-04T20:12:14Z | OWNER | I'm going to close this. Plugins can register their own CLI tools (see https://github.com/simonw/click-app) if they need to. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
315142414 | |
https://github.com/simonw/datasette/issues/221#issuecomment-754190952 | https://api.github.com/repos/simonw/datasette/issues/221 | 754190952 | MDEyOklzc3VlQ29tbWVudDc1NDE5MDk1Mg== | 9599 | 2021-01-04T20:10:51Z | 2021-01-04T20:10:51Z | OWNER | Is this still a good idea? I don't have any pressing need for it at the moment. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
315142414 | |
https://github.com/simonw/datasette/issues/221#issuecomment-754190814 | https://api.github.com/repos/simonw/datasette/issues/221 | 754190814 | MDEyOklzc3VlQ29tbWVudDc1NDE5MDgxNA== | 9599 | 2021-01-04T20:10:34Z | 2021-01-04T20:10:34Z | OWNER | For the `csvs-to-sqlite` case I'm going with `datasette insert` instead, see #1160. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
315142414 | |
https://github.com/simonw/datasette/issues/18#issuecomment-754188383 | https://api.github.com/repos/simonw/datasette/issues/18 | 754188383 | MDEyOklzc3VlQ29tbWVudDc1NDE4ODM4Mw== | 9599 | 2021-01-04T20:05:48Z | 2021-01-04T20:05:48Z | OWNER | I'm not using Sanic any more, but this is still very feasible. If I ever do it I'll write a plugin. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
267739593 | |
https://github.com/simonw/datasette/issues/103#issuecomment-754188099 | https://api.github.com/repos/simonw/datasette/issues/103 | 754188099 | MDEyOklzc3VlQ29tbWVudDc1NDE4ODA5OQ== | 9599 | 2021-01-04T20:05:14Z | 2021-01-04T20:05:14Z | OWNER | Wontfix, Cloud Run is already implemented and is a better fit for Datasette. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
274265878 | |
https://github.com/simonw/datasette/issues/913#issuecomment-754187520 | https://api.github.com/repos/simonw/datasette/issues/913 | 754187520 | MDEyOklzc3VlQ29tbWVudDc1NDE4NzUyMA== | 9599 | 2021-01-04T20:04:10Z | 2021-01-04T20:04:10Z | OWNER | That's pretty elegant: each plugin gets its own namespace and can register new settings. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
670209331 | |
https://github.com/simonw/datasette/issues/913#issuecomment-754187326 | https://api.github.com/repos/simonw/datasette/issues/913 | 754187326 | MDEyOklzc3VlQ29tbWVudDc1NDE4NzMyNg== | 9599 | 2021-01-04T20:03:50Z | 2021-01-04T20:03:50Z | OWNER | I renamed `--config` to `--setting` and changed it to work like this: datasette --setting sql_time_limit_ms 1000 Note the lack of colons. This actually makes colons cleaner to use for plugins - I could support this: datasette --setting datasette-insert:unsafe 1 | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
670209331 | |
https://github.com/simonw/datasette/issues/1111#issuecomment-754184287 | https://api.github.com/repos/simonw/datasette/issues/1111 | 754184287 | MDEyOklzc3VlQ29tbWVudDc1NDE4NDI4Nw== | 9599 | 2021-01-04T19:57:53Z | 2021-01-04T19:57:53Z | OWNER | Relevant new feature in sqlite-utils: the ability to use triggers to maintain fast counts. This optimization could help a lot here. https://github.com/simonw/sqlite-utils/issues/212 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
751195017 | |
https://github.com/simonw/datasette/issues/1164#issuecomment-754182058 | https://api.github.com/repos/simonw/datasette/issues/1164 | 754182058 | MDEyOklzc3VlQ29tbWVudDc1NDE4MjA1OA== | 9599 | 2021-01-04T19:53:31Z | 2021-01-04T19:53:31Z | OWNER | This will be helped by the new `package.json` added in #1170. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
776634318 | |
https://github.com/simonw/datasette/pull/1170#issuecomment-754181646 | https://api.github.com/repos/simonw/datasette/issues/1170 | 754181646 | MDEyOklzc3VlQ29tbWVudDc1NDE4MTY0Ng== | 9599 | 2021-01-04T19:52:40Z | 2021-01-04T19:52:40Z | OWNER | Thank you very much! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
778126516 | |
https://github.com/simonw/datasette/issues/983#issuecomment-753690280 | https://api.github.com/repos/simonw/datasette/issues/983 | 753690280 | MDEyOklzc3VlQ29tbWVudDc1MzY5MDI4MA== | 9599 | 2021-01-03T23:13:30Z | 2021-01-03T23:13:30Z | OWNER | Oh that's interesting, I hadn't thought about plugins firing events - just responding to events fired by the rest of the application. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
712260429 | |
https://github.com/simonw/sqlite-utils/issues/219#issuecomment-753671902 | https://api.github.com/repos/simonw/sqlite-utils/issues/219 | 753671902 | MDEyOklzc3VlQ29tbWVudDc1MzY3MTkwMg== | 9599 | 2021-01-03T20:31:04Z | 2021-01-03T20:32:13Z | OWNER | A `table.has_count_triggers` property. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777707544 | |
https://github.com/simonw/sqlite-utils/issues/219#issuecomment-753671235 | https://api.github.com/repos/simonw/sqlite-utils/issues/219 | 753671235 | MDEyOklzc3VlQ29tbWVudDc1MzY3MTIzNQ== | 9599 | 2021-01-03T20:25:10Z | 2021-01-03T20:25:10Z | OWNER | To detect tables, look at the names of the triggers - `{table}{counts_table}_insert` and `{table}{counts_table}_delete`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777707544 | |
https://github.com/simonw/sqlite-utils/issues/219#issuecomment-753671009 | https://api.github.com/repos/simonw/sqlite-utils/issues/219 | 753671009 | MDEyOklzc3VlQ29tbWVudDc1MzY3MTAwOQ== | 9599 | 2021-01-03T20:22:53Z | 2021-01-03T20:22:53Z | OWNER | I think this should be accompanied by a `sqlite-utils reset-counts` command. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777707544 | |
https://github.com/simonw/sqlite-utils/issues/219#issuecomment-753670833 | https://api.github.com/repos/simonw/sqlite-utils/issues/219 | 753670833 | MDEyOklzc3VlQ29tbWVudDc1MzY3MDgzMw== | 9599 | 2021-01-03T20:20:54Z | 2021-01-03T20:20:54Z | OWNER | This is a little tricky. We should assume that the existing values in the `_counts` table cannot be trusted at all when this method is called - so we should probably clear that table entirely and then re-populate it. But that means we need to figure out which tables in the database have the counts triggers defined. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777707544 | |
https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753668099 | https://api.github.com/repos/simonw/sqlite-utils/issues/215 | 753668099 | MDEyOklzc3VlQ29tbWVudDc1MzY2ODA5OQ== | 9599 | 2021-01-03T19:55:53Z | 2021-01-03T19:55:53Z | OWNER | So if you instantiate the `Database()` constructor with `use_counts_table=True` any access to the `.count` properties will go through this table - otherwise regular `count(*)` queries will be executed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777535402 | |
https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753665521 | https://api.github.com/repos/simonw/sqlite-utils/issues/215 | 753665521 | MDEyOklzc3VlQ29tbWVudDc1MzY2NTUyMQ== | 9599 | 2021-01-03T19:31:33Z | 2021-01-03T19:31:33Z | OWNER | I'm having second thoughts about this being the default behaviour. It's pretty weird. I feel like HUGE databases that need this are rare, so having it on by default doesn't make sense. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777535402 | |
https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753662490 | https://api.github.com/repos/simonw/sqlite-utils/issues/215 | 753662490 | MDEyOklzc3VlQ29tbWVudDc1MzY2MjQ5MA== | 9599 | 2021-01-03T19:05:53Z | 2021-01-03T19:05:53Z | OWNER | Idea: a `.execute_count()` method that never uses the cache. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777535402 | |
https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753661292 | https://api.github.com/repos/simonw/sqlite-utils/issues/215 | 753661292 | MDEyOklzc3VlQ29tbWVudDc1MzY2MTI5Mg== | 9599 | 2021-01-03T18:56:06Z | 2021-01-03T18:56:23Z | OWNER | Another option: on creation of the `Database()` object, check to see if the `_counts` table exists and use that as the default for a `use_counts_table` property. Also flip that property to `True` if the user calls `.enable_counts()` at any time. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777535402 | |
https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753661158 | https://api.github.com/repos/simonw/sqlite-utils/issues/215 | 753661158 | MDEyOklzc3VlQ29tbWVudDc1MzY2MTE1OA== | 9599 | 2021-01-03T18:55:16Z | 2021-01-03T18:55:16Z | OWNER | Alternative implementation: provided `db.should_trust_counts` is `True`, try running the query: ```sql select count from _counts where [table] = ? ``` If the query fails to return a result OR throws an error because the table doesn't exist, run the `count(*)` query. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777535402 | |
https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753660814 | https://api.github.com/repos/simonw/sqlite-utils/issues/215 | 753660814 | MDEyOklzc3VlQ29tbWVudDc1MzY2MDgxNA== | 9599 | 2021-01-03T18:53:05Z | 2021-01-03T18:53:05Z | OWNER | Here's the current `.count` property: https://github.com/simonw/sqlite-utils/blob/036ec6d32313487527c66dea613a3e7118b97459/sqlite_utils/db.py#L597-L609 It's implemented on `Queryable` which means it's available on both `Table` and `View` - the optimization doesn't make sense for views. I'm a bit cautious about making that property so much more complex. In order to decide if it should try the `_counts` table first it needs to know: - Should it be trusting the counts? I'm thinking a `.should_trust_counts` property on `Database` which defaults to `True` would be good - then advanced users can turn that off if they know the counts should not be trusted. - Does the `_counts` table exist? - Are the triggers defined? Then it can do the query, and if the query fails it can fall back on the `count(*)`. That's quite a lot of extra activity though. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777535402 | |
https://github.com/simonw/sqlite-utils/issues/215#issuecomment-753660379 | https://api.github.com/repos/simonw/sqlite-utils/issues/215 | 753660379 | MDEyOklzc3VlQ29tbWVudDc1MzY2MDM3OQ== | 9599 | 2021-01-03T18:50:15Z | 2021-01-03T18:50:15Z | OWNER | ```python def cached_counts(self, tables=None): sql = "select [table], count from {}".format(self._counts_table_name) if tables: sql += " where [table] in ({})".format(", ".join("?" for table in tables)) return {r[0]: r[1] for r in self.execute(sql, tables).fetchall()} ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
777535402 | |
https://github.com/simonw/sqlite-utils/issues/206#issuecomment-753659260 | https://api.github.com/repos/simonw/sqlite-utils/issues/206 | 753659260 | MDEyOklzc3VlQ29tbWVudDc1MzY1OTI2MA== | 9599 | 2021-01-03T18:42:01Z | 2021-01-03T18:42:01Z | OWNER | ``` % sqlite-utils insert blah.db blah global_power_plant_database.csv Error: Invalid JSON - use --csv for CSV or --tsv for TSV files ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
761915790 |