issue_comments
8,358 rows where author_association = "OWNER" sorted by updated_at descending
This data as json, CSV (advanced)
user 1
- simonw 6,828
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1270586897 | https://github.com/simonw/datasette/pull/1835#issuecomment-1270586897 | https://api.github.com/repos/simonw/datasette/issues/1835 | IC_kwDOBm6k_c5Lu54R | simonw 9599 | 2022-10-06T19:34:00Z | 2022-10-06T19:34:00Z | OWNER | Wow, great catch! The whole point of inspect data was to avoid this kind of expensive operation on startup so this makes total sense - I had no idea Datasette was still trying to hash a giant file every time the server started. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
use inspect data for hash and file size 1400121355 | |
1269275153 | https://github.com/simonw/datasette/issues/1480#issuecomment-1269275153 | https://api.github.com/repos/simonw/datasette/issues/1480 | IC_kwDOBm6k_c5Lp5oR | simonw 9599 | 2022-10-06T03:54:33Z | 2022-10-06T03:54:33Z | OWNER | I've been having success using Fly recently for a project which I thought would be too large for Cloud Run. I wrote about that here: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exceeding Cloud Run memory limits when deploying a 4.8G database 1015646369 | |
1267925830 | https://github.com/simonw/datasette/issues/1832#issuecomment-1267925830 | https://api.github.com/repos/simonw/datasette/issues/1832 | IC_kwDOBm6k_c5LkwNG | simonw 9599 | 2022-10-05T04:31:57Z | 2022-10-05T04:31:57Z | OWNER | Turns out this already works -
I'll add a test to demonstrate this. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
__bool__ method on Results 1397193691 | |
1267918117 | https://github.com/simonw/datasette/issues/1832#issuecomment-1267918117 | https://api.github.com/repos/simonw/datasette/issues/1832 | IC_kwDOBm6k_c5LkuUl | simonw 9599 | 2022-10-05T04:19:52Z | 2022-10-05T04:19:52Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
__bool__ method on Results 1397193691 | ||
1267709546 | https://github.com/simonw/datasette/issues/1829#issuecomment-1267709546 | https://api.github.com/repos/simonw/datasette/issues/1829 | IC_kwDOBm6k_c5Lj7Zq | simonw 9599 | 2022-10-04T23:19:24Z | 2022-10-04T23:21:07Z | OWNER | There's also a And is defined here: https://github.com/simonw/datasette/blob/4218c9cd742b79b1e3cb80878e42b7e39d16ded2/datasette/app.py#L694-L710 It's actually documented as a public method here: https://docs.datasette.io/en/stable/internals.html#await-check-visibility-actor-action-resource-none
Note that this documented method cannot actually do the right thing - because it's not being given the multiple permissions that need to be checked in order to completely answer the question. So I probably need to redesign that method a bit. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Table/database that is private due to inherited permissions does not show padlock 1396948693 | |
1267708232 | https://github.com/simonw/datasette/issues/1829#issuecomment-1267708232 | https://api.github.com/repos/simonw/datasette/issues/1829 | IC_kwDOBm6k_c5Lj7FI | simonw 9599 | 2022-10-04T23:17:36Z | 2022-10-04T23:17:36Z | OWNER | Here's the relevant code from the table page: Note how Here's the same code for the database page: And for canned query pages: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Table/database that is private due to inherited permissions does not show padlock 1396948693 | |
1264769569 | https://github.com/simonw/datasette/issues/485#issuecomment-1264769569 | https://api.github.com/repos/simonw/datasette/issues/485 | IC_kwDOBm6k_c5LYtoh | simonw 9599 | 2022-10-03T00:04:42Z | 2022-10-03T00:04:42Z | OWNER | I love these tips - tools that can compile a simple machine learning model to a SQL query! Would be pretty cool if I could bundle a model in Datasette itself as a big in-memory SQLite SQL query: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Improvements to table label detection 447469253 | |
1264753894 | https://github.com/simonw/datasette/issues/1805#issuecomment-1264753894 | https://api.github.com/repos/simonw/datasette/issues/1805 | IC_kwDOBm6k_c5LYpzm | simonw 9599 | 2022-10-02T23:02:54Z | 2022-10-02T23:02:54Z | OWNER | I'm tempted to add |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
truncate_cells_html does not work for links? 1363552780 | |
1264753725 | https://github.com/simonw/datasette/issues/1805#issuecomment-1264753725 | https://api.github.com/repos/simonw/datasette/issues/1805 | IC_kwDOBm6k_c5LYpw9 | simonw 9599 | 2022-10-02T23:02:17Z | 2022-10-02T23:02:17Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
truncate_cells_html does not work for links? 1363552780 | ||
1264753439 | https://github.com/simonw/datasette/issues/1828#issuecomment-1264753439 | https://api.github.com/repos/simonw/datasette/issues/1828 | IC_kwDOBm6k_c5LYpsf | simonw 9599 | 2022-10-02T23:01:17Z | 2022-10-02T23:01:17Z | OWNER | That change deployed and https://github-to-sqlite.dogsheep.net/github/commits now looks like this: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
word-wrap: anywhere resulting in weird display 1393903845 | |
1264738081 | https://github.com/simonw/datasette/issues/1828#issuecomment-1264738081 | https://api.github.com/repos/simonw/datasette/issues/1828 | IC_kwDOBm6k_c5LYl8h | simonw 9599 | 2022-10-02T21:34:37Z | 2022-10-02T21:34:37Z | OWNER | I'm running a build of that demo instance here (takes ~30m) https://github.com/dogsheep/github-to-sqlite/actions/runs/3170164705 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
word-wrap: anywhere resulting in weird display 1393903845 | |
1264737290 | https://github.com/simonw/datasette/issues/485#issuecomment-1264737290 | https://api.github.com/repos/simonw/datasette/issues/485 | IC_kwDOBm6k_c5LYlwK | simonw 9599 | 2022-10-02T21:29:59Z | 2022-10-02T21:29:59Z | OWNER | To clarify: the feature this issue is talking about relates to the way Datasette automatically displays foreign key relationships, for example on this page: https://github-to-sqlite.dogsheep.net/github/commits Each of those columns is a foreign key to another table. The link text that is displayed there comes from the "label column" that has either been configured or automatically detected for that other table. I wonder if this could be handled with a tiny machine learning model that's trained to help pick the best label column? Inputs to that model could include:
Output would be the most likely label column, or some indicator that no likely candidates had been found. My hunch is that this would be better solved using a few extra heuristics rather than by training a model, but it does feel like an interesting opportunity to experiment with a tiny ML model. Asked for tips about this on Twitter: https://twitter.com/simonw/status/1576680930680262658 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Improvements to table label detection 447469253 | |
1264736537 | https://github.com/simonw/datasette/issues/1805#issuecomment-1264736537 | https://api.github.com/repos/simonw/datasette/issues/1805 | IC_kwDOBm6k_c5LYlkZ | simonw 9599 | 2022-10-02T21:25:37Z | 2022-10-02T21:25:37Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
truncate_cells_html does not work for links? 1363552780 | |
1262920929 | https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1262920929 | https://api.github.com/repos/simonw/sqlite-utils/issues/297 | IC_kwDOCGYnMM5LRqTh | simonw 9599 | 2022-09-29T23:06:44Z | 2022-09-29T23:06:44Z | OWNER | Currently the only other use of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option for importing CSV data using the SQLite .import mechanism 944846776 | |
1262918833 | https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1262918833 | https://api.github.com/repos/simonw/sqlite-utils/issues/297 | IC_kwDOCGYnMM5LRpyx | simonw 9599 | 2022-09-29T23:02:52Z | 2022-09-29T23:02:52Z | OWNER | The other nice thing about having this as a separate command is that I can implement a tiny subset of the overall |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option for importing CSV data using the SQLite .import mechanism 944846776 | |
1262917059 | https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1262917059 | https://api.github.com/repos/simonw/sqlite-utils/issues/297 | IC_kwDOCGYnMM5LRpXD | simonw 9599 | 2022-09-29T22:59:28Z | 2022-09-29T22:59:28Z | OWNER | I quite like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option for importing CSV data using the SQLite .import mechanism 944846776 | |
1262915322 | https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1262915322 | https://api.github.com/repos/simonw/sqlite-utils/issues/297 | IC_kwDOCGYnMM5LRo76 | simonw 9599 | 2022-09-29T22:57:31Z | 2022-09-29T22:57:42Z | OWNER | Maybe |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option for importing CSV data using the SQLite .import mechanism 944846776 | |
1262914416 | https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1262914416 | https://api.github.com/repos/simonw/sqlite-utils/issues/297 | IC_kwDOCGYnMM5LRotw | simonw 9599 | 2022-09-29T22:56:53Z | 2022-09-29T22:56:53Z | OWNER | Potential names/designs:
Or more interestingly... what if it could accept multiple CSV files to create multiple tables?
Would still need to support creating tables with different names though. Maybe like this:
I seem to be leaning towards |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option for importing CSV data using the SQLite .import mechanism 944846776 | |
1262913145 | https://github.com/simonw/sqlite-utils/issues/297#issuecomment-1262913145 | https://api.github.com/repos/simonw/sqlite-utils/issues/297 | IC_kwDOCGYnMM5LRoZ5 | simonw 9599 | 2022-09-29T22:54:13Z | 2022-09-29T22:54:13Z | OWNER | After reviewing I'm going to implement a separate command instead. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Option for importing CSV data using the SQLite .import mechanism 944846776 | |
1260368537 | https://github.com/simonw/datasette/pull/1825#issuecomment-1260368537 | https://api.github.com/repos/simonw/datasette/issues/1825 | IC_kwDOBm6k_c5LH7KZ | simonw 9599 | 2022-09-28T04:21:18Z | 2022-09-28T04:21:18Z | OWNER | This is great, thank you very much! https://datasette--1825.org.readthedocs.build/en/1825/deploying.html#running-datasette-using-openrc |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add documentation for serving via OpenRC 1388227245 | |
1260357878 | https://github.com/simonw/datasette/issues/1826#issuecomment-1260357878 | https://api.github.com/repos/simonw/datasette/issues/1826 | IC_kwDOBm6k_c5LH4j2 | simonw 9599 | 2022-09-28T04:05:45Z | 2022-09-28T04:05:45Z | OWNER | Though now I notice that the copy right there needs to be updated to reflect the new |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
render_cell documentation example doesn't match the method signature 1388631785 | |
1260357583 | https://github.com/simonw/datasette/issues/1826#issuecomment-1260357583 | https://api.github.com/repos/simonw/datasette/issues/1826 | IC_kwDOBm6k_c5LH4fP | simonw 9599 | 2022-09-28T04:05:16Z | 2022-09-28T04:05:16Z | OWNER | This is deliberate. The Datasette plugin system allows you to specify only a subset of the parameters for a hook - in this example, only the There's a note about this at the very top of that documentation page: https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hooks
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
render_cell documentation example doesn't match the method signature 1388631785 | |
1260355224 | https://github.com/simonw/datasette/issues/526#issuecomment-1260355224 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5LH36Y | simonw 9599 | 2022-09-28T04:01:25Z | 2022-09-28T04:01:25Z | OWNER | The ultimate protection against those memory bombs is to support more streaming output formats. Related issues:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1259693536 | https://github.com/simonw/datasette/issues/526#issuecomment-1259693536 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5LFWXg | simonw 9599 | 2022-09-27T15:42:55Z | 2022-09-27T15:42:55Z | OWNER | It's interesting to note WHY the time limit works against this so well. The time limit as-implemented looks like this: https://github.com/simonw/datasette/blob/5f9f567acbc58c9fcd88af440e68034510fb5d2b/datasette/utils/init.py#L181-L201 The key here is The handler function then checks to see if too much time has transpired and conditionally cancels the query. This also doubles up as a "maximum number of operations" guard, which is what's happening when you attempt to fetch an infinite number of rows from an infinite table. That limit code could even be extended to say "exit the query after either 5s or 50,000,000 operations". I don't think that's necessary though. To be honest I'm having trouble with the idea of dropping |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1258906440 | https://github.com/simonw/datasette/issues/526#issuecomment-1258906440 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5LCWNI | simonw 9599 | 2022-09-27T03:04:37Z | 2022-09-27T03:04:37Z | OWNER | It would be really neat if we could explore this idea in a plugin, but I don't think Datasette has plugin hooks in the right place for that at the moment. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1258905781 | https://github.com/simonw/datasette/issues/526#issuecomment-1258905781 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5LCWC1 | simonw 9599 | 2022-09-27T03:03:35Z | 2022-09-27T03:03:47Z | OWNER | Yes good point, the time limit does already protect against that. I've been contemplating a permissioned-users-only relaxation of that time limit too, and I got that idea mixed up with this one in my head. On that basis maybe this feature would be safe after all? Would need to do some testing, but it may be that the existing time limit provides enough protection here already. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1258864140 | https://github.com/simonw/datasette/issues/526#issuecomment-1258864140 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5LCL4M | simonw 9599 | 2022-09-27T01:55:32Z | 2022-09-27T01:55:32Z | OWNER | That recursive query is a great example of the kind of thing having a maximum row limit protects against. Imagine if Datasette CSVs did allow unlimited retrievals. Someone could hit the CSV endpoint for that recursive query and tie up Datasette's SQL connection effectively forever. Even if this feature becomes a permission-guarded thing we still need to take that case into account. At the very least it would be good if the query could be cancelled if the client disconnects - so if someone accidentally starts an infinite query they can cancel the request and free up the server resources. It might be a good idea to implement a page that shows "currently running" queries and allows users with the right permission to terminate them from that page. Another option: a "limit of last resource" - either a very high row limit (10,000,000 perhaps) or even a time limit, saying that all queries will be cancelled if they take longer than thirty minutes or similar. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1258860845 | https://github.com/simonw/datasette/issues/526#issuecomment-1258860845 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5LCLEt | simonw 9599 | 2022-09-27T01:48:31Z | 2022-09-27T01:50:01Z | OWNER | The protection is supposed to be from this line:
SQLite and the ```pycon
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1258846992 | https://github.com/simonw/datasette/issues/526#issuecomment-1258846992 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5LCHsQ | simonw 9599 | 2022-09-27T01:21:41Z | 2022-09-27T01:21:41Z | OWNER | My main concern here is that public Datasette instances could easily have all of their available database connections consumed by long-running queries - either accidentally or deliberately. I do totally understand the need for this feature though. I think it can absolutely make sense provided it's protected by authentication and permissions. Maybe even limit the number of concurrent downloads at once such that there's always at least one database connection free for other requests. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1258828705 | https://github.com/simonw/datasette/pull/1823#issuecomment-1258828705 | https://api.github.com/repos/simonw/datasette/issues/1823 | IC_kwDOBm6k_c5LCDOh | simonw 9599 | 2022-09-27T00:45:46Z | 2022-09-27T00:45:46Z | OWNER | Also need to do a bit more of an audit to see if there is anywhere else that this style should be applied. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Keyword-only arguments for a bunch of internal methods 1386917344 | |
1258828509 | https://github.com/simonw/datasette/pull/1823#issuecomment-1258828509 | https://api.github.com/repos/simonw/datasette/issues/1823 | IC_kwDOBm6k_c5LCDLd | simonw 9599 | 2022-09-27T00:45:26Z | 2022-09-27T00:45:26Z | OWNER | I should update the documentation to reflect this change. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Keyword-only arguments for a bunch of internal methods 1386917344 | |
1258827688 | https://github.com/simonw/datasette/issues/1822#issuecomment-1258827688 | https://api.github.com/repos/simonw/datasette/issues/1822 | IC_kwDOBm6k_c5LCC-o | simonw 9599 | 2022-09-27T00:44:04Z | 2022-09-27T00:44:04Z | OWNER | I'll do this in a PR. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch to keyword-only arguments for a bunch of internal methods 1386854246 | |
1258818028 | https://github.com/simonw/datasette/issues/1817#issuecomment-1258818028 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5LCAns | simonw 9599 | 2022-09-27T00:27:53Z | 2022-09-27T00:27:53Z | OWNER | Made a start on this: ```diff diff --git a/datasette/hookspecs.py b/datasette/hookspecs.py index 34e19664..fe0971e5 100644 --- a/datasette/hookspecs.py +++ b/datasette/hookspecs.py @@ -31,25 +31,29 @@ def prepare_jinja2_environment(env, datasette): @hookspec -def extra_css_urls(template, database, table, columns, view_name, request, datasette): +def extra_css_urls( + template, database, table, columns, sql, params, view_name, request, datasette +): """Extra CSS URLs added by this plugin""" @hookspec -def extra_js_urls(template, database, table, columns, view_name, request, datasette): +def extra_js_urls( + template, database, table, columns, sql, params, view_name, request, datasette +): """Extra JavaScript URLs added by this plugin""" @hookspec def extra_body_script( - template, database, table, columns, view_name, request, datasette + template, database, table, columns, sql, params, view_name, request, datasette ): """Extra JavaScript code to be included in <script> at bottom of body""" @hookspec
def extra_template_vars(
- template, database, table, columns, view_name, request, datasette
+ template, database, table, columns, sql, params, view_name, request, datasette
):
"""Extra template variables to be made available to the template - can return dict or callable or awaitable"""
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
1258760299 | https://github.com/simonw/datasette/issues/1822#issuecomment-1258760299 | https://api.github.com/repos/simonw/datasette/issues/1822 | IC_kwDOBm6k_c5LByhr | simonw 9599 | 2022-09-26T23:25:12Z | 2022-09-26T23:25:55Z | OWNER | A start: ```diff diff --git a/datasette/utils/asgi.py b/datasette/utils/asgi.py index 8a2fa060..41ade961 100644 --- a/datasette/utils/asgi.py +++ b/datasette/utils/asgi.py @@ -118,7 +118,7 @@ class Request: return dict(parse_qsl(body.decode("utf-8"), keep_blank_values=True))
-async def asgi_send_json(send, info, status=200, headers=None): +async def asgi_send_json(send, info, *, status=200, headers=None): headers = headers or {} await asgi_send( send, @@ -215,7 +215,7 @@ async def asgi_send_json(send, info, status=200, headers=None): ) -async def asgi_send_html(send, html, status=200, headers=None): +async def asgi_send_html(send, html, *, status=200, headers=None): headers = headers or {} await asgi_send( send, @@ -226,7 +226,7 @@ async def asgi_send_html(send, html, status=200, headers=None): ) -async def asgi_send_redirect(send, location, status=302): +async def asgi_send_redirect(send, location, *, status=302): await asgi_send( send, "", @@ -236,12 +236,12 @@ async def asgi_send_redirect(send, location, status=302): ) -async def asgi_send(send, content, status, headers=None, content_type="text/plain"): +async def asgi_send(send, content, status, *, headers=None, content_type="text/plain"): await asgi_start(send, status, headers, content_type) await send({"type": "http.response.body", "body": content.encode("utf-8")}) -async def asgi_start(send, status, headers=None, content_type="text/plain"): +async def asgi_start(send, status, *, headers=None, content_type="text/plain"): headers = headers or {} # Remove any existing content-type header headers = {k: v for k, v in headers.items() if k.lower() != "content-type"} @@ -259,7 +259,7 @@ async def asgi_start(send, status, headers=None, content_type="text/plain"): async def asgi_send_file( - send, filepath, filename=None, content_type=None, chunk_size=4096, headers=None + send, filepath, filename=None, *, content_type=None, chunk_size=4096, headers=None ): headers = headers or {} if filename: @@ -284,7 +284,7 @@ async def asgi_send_file( ) -def asgi_static(root_path, chunk_size=4096, headers=None, content_type=None): +def asgi_static(root_path, *, chunk_size=4096, headers=None, content_type=None): root_path = Path(root_path)
@@ -313,7 +313,7 @@ def asgi_static(root_path, chunk_size=4096, headers=None, content_type=None): class Response: - def init(self, body=None, status=200, headers=None, content_type="text/plain"): + def init(self, body=None, , status=200, headers=None, content_type="text/plain"): self.body = body self.status = status self.headers = headers or {} @@ -346,6 +346,7 @@ class Response: self, key, value="", + , max_age=None, expires=None, path="/", @@ -374,7 +375,7 @@ class Response: self._set_cookie_headers.append(cookie.output(header="").strip())
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch to keyword-only arguments for a bunch of internal methods 1386854246 | |
1258757544 | https://github.com/simonw/datasette/issues/1822#issuecomment-1258757544 | https://api.github.com/repos/simonw/datasette/issues/1822 | IC_kwDOBm6k_c5LBx2o | simonw 9599 | 2022-09-26T23:21:23Z | 2022-09-26T23:21:23Z | OWNER | Everything on https://docs.datasette.io/en/stable/internals.html that uses keyword arguments should do this I think. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch to keyword-only arguments for a bunch of internal methods 1386854246 | |
1258756231 | https://github.com/simonw/datasette/issues/1817#issuecomment-1258756231 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5LBxiH | simonw 9599 | 2022-09-26T23:19:34Z | 2022-09-26T23:19:34Z | OWNER | This is a good idea - it's something I should do before Datasette 1.0. I was a tiny bit worried about compatibility (Datasette is 3.7+) but it looks like they have been in Python since 3.0! |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
1258754105 | https://github.com/simonw/datasette/issues/1819#issuecomment-1258754105 | https://api.github.com/repos/simonw/datasette/issues/1819 | IC_kwDOBm6k_c5LBxA5 | simonw 9599 | 2022-09-26T23:16:15Z | 2022-09-26T23:16:15Z | OWNER | Demo: https://latest.datasette.io/_memory?sql=with+recursive+counter(x)+as+(%0D%0A++select+0%0D%0A++++union%0D%0A++select+x+%2B+1+from+counter%0D%0A)%2C%0D%0Ablah+as+(select++from+counter+limit+5000000)%0D%0Aselect+count()+from+blah |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Preserve query on timeout 1385026210 | |
1258746600 | https://github.com/simonw/datasette/issues/1819#issuecomment-1258746600 | https://api.github.com/repos/simonw/datasette/issues/1819 | IC_kwDOBm6k_c5LBvLo | simonw 9599 | 2022-09-26T23:05:40Z | 2022-09-26T23:05:40Z | OWNER | Implementing it like this, so at least you can copy and paste the SQL query back out again: I'm not doing a full textarea because this error can be raised in multiple places, including on the table page itself. It's not just an error associated with the manual query page. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Preserve query on timeout 1385026210 | |
1258738435 | https://github.com/simonw/datasette/issues/1819#issuecomment-1258738435 | https://api.github.com/repos/simonw/datasette/issues/1819 | IC_kwDOBm6k_c5LBtMD | simonw 9599 | 2022-09-26T22:52:19Z | 2022-09-26T22:52:19Z | OWNER | This is a good idea. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Preserve query on timeout 1385026210 | |
1258735747 | https://github.com/simonw/datasette/issues/1818#issuecomment-1258735747 | https://api.github.com/repos/simonw/datasette/issues/1818 | IC_kwDOBm6k_c5LBsiD | simonw 9599 | 2022-09-26T22:47:59Z | 2022-09-26T22:47:59Z | OWNER | Another option here is to tie into a feature I built in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Setting to turn off table row counts entirely 1384549993 | |
1258735283 | https://github.com/simonw/datasette/issues/1818#issuecomment-1258735283 | https://api.github.com/repos/simonw/datasette/issues/1818 | IC_kwDOBm6k_c5LBsaz | simonw 9599 | 2022-09-26T22:47:19Z | 2022-09-26T22:47:19Z | OWNER | That's a really interesting idea: for a lot of databases (those made out of straight imports from CSV) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Setting to turn off table row counts entirely 1384549993 | |
1258697384 | https://github.com/simonw/sqlite-utils/issues/491#issuecomment-1258697384 | https://api.github.com/repos/simonw/sqlite-utils/issues/491 | IC_kwDOCGYnMM5LBjKo | simonw 9599 | 2022-09-26T22:12:45Z | 2022-09-26T22:12:45Z | OWNER | That feels like a slightly different command to me - maybe |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to merge databases and tables 1383646615 | |
1258692555 | https://github.com/simonw/datasette/issues/1821#issuecomment-1258692555 | https://api.github.com/repos/simonw/datasette/issues/1821 | IC_kwDOBm6k_c5LBh_L | simonw 9599 | 2022-09-26T22:06:39Z | 2022-09-26T22:06:39Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Release Datasette 0.63a0 1386734383 | ||
1258521333 | https://github.com/simonw/sqlite-utils/issues/494#issuecomment-1258521333 | https://api.github.com/repos/simonw/sqlite-utils/issues/494 | IC_kwDOCGYnMM5LA4L1 | simonw 9599 | 2022-09-26T19:32:36Z | 2022-09-26T19:32:36Z | OWNER | Tweeted about it too: https://twitter.com/simonw/status/1574481628507668480 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use Just 1386593843 | |
1258516872 | https://github.com/simonw/sqlite-utils/issues/494#issuecomment-1258516872 | https://api.github.com/repos/simonw/sqlite-utils/issues/494 | IC_kwDOCGYnMM5LA3GI | simonw 9599 | 2022-09-26T19:28:36Z | 2022-09-26T19:28:36Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Document how to use Just 1386593843 | ||
1258479462 | https://github.com/simonw/sqlite-utils/issues/483#issuecomment-1258479462 | https://api.github.com/repos/simonw/sqlite-utils/issues/483 | IC_kwDOCGYnMM5LAt9m | simonw 9599 | 2022-09-26T19:04:29Z | 2022-09-26T19:04:43Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`sqlite-utils install` command 1363765916 | ||
1258476455 | https://github.com/simonw/sqlite-utils/issues/493#issuecomment-1258476455 | https://api.github.com/repos/simonw/sqlite-utils/issues/493 | IC_kwDOCGYnMM5LAtOn | simonw 9599 | 2022-09-26T19:01:49Z | 2022-09-26T19:01:49Z | OWNER | I tried the tips in https://stackoverflow.com/questions/15258831/how-to-handle-two-dashes-in-rest (not the settings change though, because I might want smart quotes elsewhere) and they didn't work. Maybe I should disable smart quotes entirely? I feel like there should be an escaping trick that works here though. I tried |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Tiny typographical error in install/uninstall docs 1386562662 | |
1258451968 | https://github.com/simonw/sqlite-utils/issues/483#issuecomment-1258451968 | https://api.github.com/repos/simonw/sqlite-utils/issues/483 | IC_kwDOCGYnMM5LAnQA | simonw 9599 | 2022-09-26T18:37:54Z | 2022-09-26T18:40:41Z | OWNER | The implementation of this can be an almost exact copy of Datasette's, which was added in this commit: https://github.com/simonw/datasette/commit/01fe5b740171bfaea3752fc5754431dac53777e3 Current code for that is here: https://github.com/simonw/datasette/blob/0.62/datasette/cli.py#L319-L340 - which is improved to use the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`sqlite-utils install` command 1363765916 | |
1258450447 | https://github.com/simonw/sqlite-utils/issues/491#issuecomment-1258450447 | https://api.github.com/repos/simonw/sqlite-utils/issues/491 | IC_kwDOCGYnMM5LAm4P | simonw 9599 | 2022-09-26T18:36:23Z | 2022-09-26T18:36:23Z | OWNER | This is also the kind of feature that would need to express itself in both the Python library and the CLI utility. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to merge databases and tables 1383646615 | |
1258449887 | https://github.com/simonw/sqlite-utils/issues/491#issuecomment-1258449887 | https://api.github.com/repos/simonw/sqlite-utils/issues/491 | IC_kwDOCGYnMM5LAmvf | simonw 9599 | 2022-09-26T18:35:50Z | 2022-09-26T18:35:50Z | OWNER | This is a really interesting idea. I'm nervous about needing to set the rules for how duplicate tables should be merged though. This feels like a complex topic - one where there isn't necessarily an obviously "correct" way of doing it, but where different problems that people are solving might need different merging approaches. Likewise, merging isn't just a database-to-database thing at that point - I could see a need for merging two tables using similar rules to those used for merging two databases. So I think I'd want to have some good concrete use-cases in mind before trying to design how something like this should work. Will leave this thread open for people to drop those in! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to merge databases and tables 1383646615 | |
1258446128 | https://github.com/simonw/sqlite-utils/issues/492#issuecomment-1258446128 | https://api.github.com/repos/simonw/sqlite-utils/issues/492 | IC_kwDOCGYnMM5LAl0w | simonw 9599 | 2022-09-26T18:32:14Z | 2022-09-26T18:33:19Z | OWNER | This idea would make more sense if there was a good mechanism to say "run the conversion script held in this file" as opposed to passing it as an option. This would also make having to remember bash escaping rules (see tip) much easier!
Maybe |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Idea: ability to pass extra variables to `--convert` scripts 1386530156 | |
1258437060 | https://github.com/simonw/sqlite-utils/issues/490#issuecomment-1258437060 | https://api.github.com/repos/simonw/sqlite-utils/issues/490 | IC_kwDOCGYnMM5LAjnE | simonw 9599 | 2022-09-26T18:24:44Z | 2022-09-26T18:24:44Z | OWNER | Just saw your great write-up on this: https://jeqo.github.io/notes/2022-09-24-ingest-logs-sqlite/ |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
Ability to insert multi-line files 1382457780 | |
1256662785 | https://github.com/simonw/datasette/issues/1817#issuecomment-1256662785 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5K5ycB | simonw 9599 | 2022-09-23T20:53:21Z | 2022-09-23T20:53:21Z | OWNER | Maybe the signature for that method should be:
Those would then be passed when specific views call So yet another change that's blocked on fixing that long-running weird piece of technical debt: - https://github.com/simonw/datasette/issues/1518 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
1256659788 | https://github.com/simonw/datasette/issues/1817#issuecomment-1256659788 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5K5xtM | simonw 9599 | 2022-09-23T20:49:22Z | 2022-09-23T20:49:22Z | OWNER | Implementation challenge: all four of those hooks are called inside the So I would have to pull the Might be an opportunity to clean up this hack: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
1256652548 | https://github.com/simonw/datasette/issues/1817#issuecomment-1256652548 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5K5v8E | simonw 9599 | 2022-09-23T20:41:32Z | 2022-09-23T20:41:32Z | OWNER | Which plugin hooks should take
And maybe these:
I'll start by implementing the first set, then I'll think further about those "maybes". |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
1256650449 | https://github.com/simonw/datasette/issues/1817#issuecomment-1256650449 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5K5vbR | simonw 9599 | 2022-09-23T20:38:53Z | 2022-09-23T20:38:53Z | OWNER | I've wanted something like this in the past too. I think the thing to do here might be to add While I'm working on this: https://docs.datasette.io/en/0.62/plugin_hooks.html#register-output-renderer-datasette output renderer functions take |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
1256428818 | https://github.com/simonw/sqlite-utils/issues/490#issuecomment-1256428818 | https://api.github.com/repos/simonw/sqlite-utils/issues/490 | IC_kwDOCGYnMM5K45US | simonw 9599 | 2022-09-23T16:37:58Z | 2022-09-23T16:38:35Z | OWNER | It should be possible to achieve this with the Given an example like this in r = re.compile(r'^(?P<datetime>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}):(?P<log>.*)', re.MULTILINE) def convert(text):
return [m.groupdict() for m in r.finditer(text)]
"
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to insert multi-line files 1382457780 | |
1255341690 | https://github.com/simonw/sqlite-utils/issues/358#issuecomment-1255341690 | https://api.github.com/repos/simonw/sqlite-utils/issues/358 | IC_kwDOCGYnMM5K0v56 | simonw 9599 | 2022-09-22T17:35:23Z | 2022-09-22T17:35:23Z | OWNER | Make me think also that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Support for CHECK constraints 1082651698 | |
1255340974 | https://github.com/simonw/sqlite-utils/issues/358#issuecomment-1255340974 | https://api.github.com/repos/simonw/sqlite-utils/issues/358 | IC_kwDOCGYnMM5K0vuu | simonw 9599 | 2022-09-22T17:34:45Z | 2022-09-22T17:34:45Z | OWNER | A few other recipes off the top of my head:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Support for CHECK constraints 1082651698 | |
1255333969 | https://github.com/simonw/sqlite-utils/issues/358#issuecomment-1255333969 | https://api.github.com/repos/simonw/sqlite-utils/issues/358 | IC_kwDOCGYnMM5K0uBR | simonw 9599 | 2022-09-22T17:29:09Z | 2022-09-22T17:29:09Z | OWNER | Quick demo of a check constraint for JSON validation:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Support for CHECK constraints 1082651698 | |
1255332217 | https://github.com/simonw/sqlite-utils/issues/358#issuecomment-1255332217 | https://api.github.com/repos/simonw/sqlite-utils/issues/358 | IC_kwDOCGYnMM5K0tl5 | simonw 9599 | 2022-09-22T17:27:34Z | 2022-09-22T17:27:34Z | OWNER | I've been thinking about this more recently. I think the first place to explore these will be in the Relevant docs: https://www.sqlite.org/lang_createtable.html#check_constraints
Something like this:
Where Then can bundle a bunch of other pre-baked recipes, but also support the following:
The besign reason for the Detecting |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Support for CHECK constraints 1082651698 | |
1254033981 | https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1254033981 | https://api.github.com/repos/simonw/sqlite-utils/issues/488 | IC_kwDOCGYnMM5Kvwo9 | simonw 9599 | 2022-09-21T17:49:32Z | 2022-09-21T17:50:10Z | OWNER | It looks like SQLite has a We need to only apply that function to columns that we know to be of type integer or float though - text columns containing empty strings should not be rewritten to null. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657 | |
1254032378 | https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1254032378 | https://api.github.com/repos/simonw/sqlite-utils/issues/488 | IC_kwDOCGYnMM5KvwP6 | simonw 9599 | 2022-09-21T17:47:54Z | 2022-09-21T17:47:54Z | OWNER | New tests should go in: https://github.com/simonw/sqlite-utils/blob/main/tests/test_transform.py I think the implementation fix needs to go near here: https://github.com/simonw/sqlite-utils/blob/0b315d3fa83c1584eaeec32f24912898621e437a/sqlite_utils/db.py#L1770-L1775 The trick is going to be teaching that generated SQL to know which columns are |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657 | |
1254029808 | https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1254029808 | https://api.github.com/repos/simonw/sqlite-utils/issues/488 | IC_kwDOCGYnMM5Kvvnw | simonw 9599 | 2022-09-21T17:45:20Z | 2022-09-21T17:45:41Z | OWNER | No, I'm going to say that this is a bug - it's WEIRD having a So no need for a new option here - just fixing the bug is sensible. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657 | |
1251724180 | https://github.com/simonw/datasette/issues/1816#issuecomment-1251724180 | https://api.github.com/repos/simonw/datasette/issues/1816 | IC_kwDOBm6k_c5Km8uU | simonw 9599 | 2022-09-20T01:13:05Z | 2022-09-20T01:13:05Z | OWNER | Oops, that has a bug:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Validate settings.json on startup in configuration directory mode 1378640768 | |
1251682970 | https://github.com/simonw/datasette/issues/1816#issuecomment-1251682970 | https://api.github.com/repos/simonw/datasette/issues/1816 | IC_kwDOBm6k_c5Kmyqa | simonw 9599 | 2022-09-19T23:44:54Z | 2022-09-19T23:44:54Z | OWNER | I was going to add type validation too, but that's actually a bit tricky because the logic for that currently lives in Click option parsing here: https://github.com/simonw/datasette/blob/ddc999ad1296e8c69cffede3e367dda059b8adad/datasette/cli.py#L71-L88 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Validate settings.json on startup in configuration directory mode 1378640768 | |
1251677554 | https://github.com/simonw/datasette/issues/1814#issuecomment-1251677554 | https://api.github.com/repos/simonw/datasette/issues/1814 | IC_kwDOBm6k_c5KmxVy | simonw 9599 | 2022-09-19T23:35:06Z | 2022-09-19T23:35:06Z | OWNER | It might have been useful for Datasette to show an error when started against a |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Static files not served 1378495690 | |
1251677220 | https://github.com/simonw/datasette/issues/1814#issuecomment-1251677220 | https://api.github.com/repos/simonw/datasette/issues/1814 | IC_kwDOBm6k_c5KmxQk | simonw 9599 | 2022-09-19T23:34:30Z | 2022-09-19T23:34:30Z | OWNER | The The
|
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0 } |
Static files not served 1378495690 | |
1249990033 | https://github.com/simonw/datasette/issues/1809#issuecomment-1249990033 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KgVWR | simonw 9599 | 2022-09-17T03:39:05Z | 2022-09-17T03:39:05Z | OWNER | New docs section on the need to call |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1249987643 | https://github.com/simonw/datasette/issues/1809#issuecomment-1249987643 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KgUw7 | simonw 9599 | 2022-09-17T03:19:24Z | 2022-09-17T03:19:24Z | OWNER | In looking at the documentation on writing tests, there are a lot of examples like this:
Since it's safe to call that function multiple times, I'm going to have |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1249986079 | https://github.com/simonw/datasette/issues/1809#issuecomment-1249986079 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KgUYf | simonw 9599 | 2022-09-17T03:07:24Z | 2022-09-17T03:07:24Z | OWNER | Datasette's own tests started to break because calls to the
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1249985971 | https://github.com/simonw/datasette/issues/1809#issuecomment-1249985971 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KgUWz | simonw 9599 | 2022-09-17T03:06:32Z | 2022-09-17T03:06:32Z | OWNER | This is likely going to cause some tests in plugins to break, but I'm OK with that - I'll fix them as I find them once this release is out. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1249985741 | https://github.com/simonw/datasette/issues/1809#issuecomment-1249985741 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KgUTN | simonw 9599 | 2022-09-17T03:04:51Z | 2022-09-17T03:04:51Z | OWNER | I'm going to throw an error in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1249746777 | https://github.com/simonw/datasette/pull/1812#issuecomment-1249746777 | https://api.github.com/repos/simonw/datasette/issues/1812 | IC_kwDOBm6k_c5KfZ9Z | simonw 9599 | 2022-09-16T19:50:45Z | 2022-09-16T19:50:45Z | OWNER | Main difference I can see: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump furo from 2022.6.21 to 2022.9.15 1375930971 | |
1249745637 | https://github.com/simonw/datasette/pull/1812#issuecomment-1249745637 | https://api.github.com/repos/simonw/datasette/issues/1812 | IC_kwDOBm6k_c5KfZrl | simonw 9599 | 2022-09-16T19:49:12Z | 2022-09-16T19:49:12Z | OWNER | Preview looks good. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump furo from 2022.6.21 to 2022.9.15 1375930971 | |
1248621072 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248621072 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5KbHIQ | simonw 9599 | 2022-09-15T20:56:09Z | 2022-09-15T20:56:09Z | OWNER | Prototype so far:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248591268 | https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248591268 | https://api.github.com/repos/simonw/sqlite-utils/issues/486 | IC_kwDOCGYnMM5Ka_2k | simonw 9599 | 2022-09-15T20:36:02Z | 2022-09-15T20:40:03Z | OWNER | I had a big CSV file lying around, I converted it to other formats like this:
Then tested the progress bar like this:
Output:
And for regular JSON:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
progressbar for inserts/upserts of all fileformats, closes #485 1366512990 | |
1248597643 | https://github.com/simonw/sqlite-utils/issues/485#issuecomment-1248597643 | https://api.github.com/repos/simonw/sqlite-utils/issues/485 | IC_kwDOCGYnMM5KbBaL | simonw 9599 | 2022-09-15T20:39:39Z | 2022-09-15T20:39:52Z | OWNER | A note from PR #486: https://github.com/simonw/sqlite-utils/issues/486#issuecomment-1248591268_
I decided to land this anyway. If a streaming JSON parser is added later it will start to work. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Progressbar not shown when inserting/upserting jsonlines file 1366423176 | |
1248593835 | https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248593835 | https://api.github.com/repos/simonw/sqlite-utils/issues/486 | IC_kwDOCGYnMM5KbAer | simonw 9599 | 2022-09-15T20:37:14Z | 2022-09-15T20:37:14Z | OWNER | I'm going to land this anyway. The lack of a streaming JSON parser is a separate issue, I don't think it should block landing this improvement. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
progressbar for inserts/upserts of all fileformats, closes #485 1366512990 | |
1248582147 | https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248582147 | https://api.github.com/repos/simonw/sqlite-utils/issues/486 | IC_kwDOCGYnMM5Ka9oD | simonw 9599 | 2022-09-15T20:29:17Z | 2022-09-15T20:29:17Z | OWNER | This looks good to me. I need to run some manual tests before merging (it's a good sign that the automated tests pass though). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
progressbar for inserts/upserts of all fileformats, closes #485 1366512990 | |
1248568775 | https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248568775 | https://api.github.com/repos/simonw/sqlite-utils/issues/486 | IC_kwDOCGYnMM5Ka6XH | simonw 9599 | 2022-09-15T20:16:14Z | 2022-09-15T20:16:14Z | OWNER | https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#using-the-python-version-input says can set the full version:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
progressbar for inserts/upserts of all fileformats, closes #485 1366512990 | |
1248567323 | https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248567323 | https://api.github.com/repos/simonw/sqlite-utils/issues/486 | IC_kwDOCGYnMM5Ka6Ab | simonw 9599 | 2022-09-15T20:14:45Z | 2022-09-15T20:14:45Z | OWNER | There's a fix for For the moment looks like pinning to Python 3.10.6 could help. Need to figure out how to do that in GitHub Actions though. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
progressbar for inserts/upserts of all fileformats, closes #485 1366512990 | |
1248565396 | https://github.com/simonw/sqlite-utils/pull/486#issuecomment-1248565396 | https://api.github.com/repos/simonw/sqlite-utils/issues/486 | IC_kwDOCGYnMM5Ka5iU | simonw 9599 | 2022-09-15T20:12:50Z | 2022-09-15T20:12:50Z | OWNER | Annoying
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
progressbar for inserts/upserts of all fileformats, closes #485 1366512990 | |
1248522618 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248522618 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5KavF6 | simonw 9599 | 2022-09-15T19:29:20Z | 2022-09-15T19:29:20Z | OWNER | I think refactoring |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248512739 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248512739 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5Kasrj | simonw 9599 | 2022-09-15T19:18:24Z | 2022-09-15T19:21:01Z | OWNER | Why doesn't https://github.com/simonw/sqlite-utils/issues/279#issuecomment-864207841 says:
Maybe I forgot to do that? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248501824 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248501824 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5KaqBA | simonw 9599 | 2022-09-15T19:10:48Z | 2022-09-15T19:10:48Z | OWNER | This feels pretty good:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248484094 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248484094 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5Kalr- | simonw 9599 | 2022-09-15T18:56:31Z | 2022-09-15T18:56:31Z | OWNER | Actually I quite like
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248481303 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248481303 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5KalAX | simonw 9599 | 2022-09-15T18:54:30Z | 2022-09-15T18:55:14Z | OWNER | Maybe this would make more sense as a mechanism where you can say "Use the data in the key called X" - but there's a special option for "figure out that key automatically". The syntax then could be:
Or for automatic detection:
Could also go with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248479485 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248479485 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5Kakj9 | simonw 9599 | 2022-09-15T18:52:52Z | 2022-09-15T18:53:45Z | OWNER | The most similar option I have at the moment is probably
Those are all bad. Another option: introduce a new explicit format for it. Right now the explicit formats you can use are: So I could add a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248475718 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248475718 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5KajpG | simonw 9599 | 2022-09-15T18:49:05Z | 2022-09-15T18:49:53Z | OWNER | Here's how I used my prototype to build that Gist:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248474806 | https://github.com/simonw/sqlite-utils/issues/489#issuecomment-1248474806 | https://api.github.com/repos/simonw/sqlite-utils/issues/489 | IC_kwDOCGYnMM5Kaja2 | simonw 9599 | 2022-09-15T18:48:09Z | 2022-09-15T18:48:09Z | OWNER | Built a prototype of this that works really well:
One problem though: right now, if you do this So I probably need some kind of opt-in mechanism for this. And I need a good name for it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to load JSON records held in a file with a single top level key that is a list of objects 1374939463 | |
1248290151 | https://github.com/simonw/datasette/issues/1810#issuecomment-1248290151 | https://api.github.com/repos/simonw/datasette/issues/1810 | IC_kwDOBm6k_c5KZ2Vn | simonw 9599 | 2022-09-15T15:51:04Z | 2022-09-15T15:51:25Z | OWNER | I could prototype this idea as a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Featured table(s) on the homepage 1374626873 | |
1248289857 | https://github.com/simonw/datasette/issues/1810#issuecomment-1248289857 | https://api.github.com/repos/simonw/datasette/issues/1810 | IC_kwDOBm6k_c5KZ2RB | simonw 9599 | 2022-09-15T15:50:46Z | 2022-09-15T15:50:46Z | OWNER | Idea: allow the user to specify one or more featured tables. Each table is then shown as a summary on the homepage - with the total number of rows and the first 5 rows. If the table has search configured there's a search box too. If the instance has only one database with only one table (excluding hidden tables) it gets featured automatically perhaps (maybe with a way to opt-out of that if you want to). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Featured table(s) on the homepage 1374626873 | |
1248187089 | https://github.com/simonw/datasette/issues/1810#issuecomment-1248187089 | https://api.github.com/repos/simonw/datasette/issues/1810 | IC_kwDOBm6k_c5KZdLR | simonw 9599 | 2022-09-15T14:31:36Z | 2022-09-15T14:31:36Z | OWNER | Twitter conversation that inspired this issue: https://twitter.com/psychemedia/status/1570410108785684481 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Featured table(s) on the homepage 1374626873 | |
1247317941 | https://github.com/simonw/datasette/issues/1809#issuecomment-1247317941 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KWI-1 | simonw 9599 | 2022-09-14T21:24:43Z | 2022-09-14T21:24:43Z | OWNER | It looks like Datasette Lite does NOT invoke that method, which is likely a bug: https://github.com/simonw/datasette-lite/blob/e7ccaf621b3cdf613ebaf544304d387f2af32edf/webworker.js#L103-L110 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1247316715 | https://github.com/simonw/datasette/issues/1809#issuecomment-1247316715 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KWIrr | simonw 9599 | 2022-09-14T21:23:10Z | 2022-09-14T21:23:10Z | OWNER | It might be good to have Datasette LOUDLY fail if you attempt to use it without calling |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1247316097 | https://github.com/simonw/datasette/issues/1809#issuecomment-1247316097 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KWIiB | simonw 9599 | 2022-09-14T21:22:24Z | 2022-09-14T21:22:24Z | OWNER | It looks like this is the only place that calls
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1247314352 | https://github.com/simonw/datasette/issues/1809#issuecomment-1247314352 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KWIGw | simonw 9599 | 2022-09-14T21:20:12Z | 2022-09-14T21:20:12Z | OWNER | The reason to support That's exactly what ```python @hookimpl def startup(datasette): datasette._edit_templates = {}
``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1247313134 | https://github.com/simonw/datasette/issues/1809#issuecomment-1247313134 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KWHzu | simonw 9599 | 2022-09-14T21:18:46Z | 2022-09-14T21:18:46Z | OWNER |
Which is inside the To implement an But I'm not 100% confident that is always executed at the right time to ensure the Jinja environment is properly configured? I think it is, but would need to make sure. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 | |
1247311275 | https://github.com/simonw/datasette/issues/1809#issuecomment-1247311275 | https://api.github.com/repos/simonw/datasette/issues/1809 | IC_kwDOBm6k_c5KWHWr | simonw 9599 | 2022-09-14T21:16:32Z | 2022-09-14T21:16:32Z | OWNER | It should also implement the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`prepare_jinja2_environment()` hook should take `datasette` argument 1373595927 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue >30