issue_comments
8,883 rows where user = 9599 sorted by updated_at descending
This data as json, CSV (advanced)
user 1
- simonw · 6,677 ✖
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1076652046 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076652046 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALGgO | simonw 9599 | 2022-03-23T18:02:30Z | 2022-03-23T18:02:30Z | OWNER | Two new things to add to the release notes from https://github.com/simonw/datasette/compare/0.61a0...main - https://github.com/simonw/datasette/issues/1678 - https://github.com/simonw/datasette/issues/1675 (now also a documented API) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | |
1076647495 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076647495 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALFZH | simonw 9599 | 2022-03-23T17:58:16Z | 2022-03-23T17:58:16Z | OWNER | I think the release notes are fine, but they need an opening paragraph highlighting the changes that are most likely to break backwards compatibility. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | |
1076645636 | https://github.com/simonw/datasette/pull/1574#issuecomment-1076645636 | https://api.github.com/repos/simonw/datasette/issues/1574 | IC_kwDOBm6k_c5ALE8E | simonw 9599 | 2022-03-23T17:56:35Z | 2022-03-23T17:56:35Z | OWNER | I'd actually like to switch to slim as the default - I think Datasette should ship the smallest possible container that can still support extra packages being installed using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
introduce new option for datasette package to use a slim base image 1084193403 | |
1076644362 | https://github.com/simonw/datasette/pull/1665#issuecomment-1076644362 | https://api.github.com/repos/simonw/datasette/issues/1665 | IC_kwDOBm6k_c5ALEoK | simonw 9599 | 2022-03-23T17:55:39Z | 2022-03-23T17:55:39Z | OWNER | Thanks for the PR - I spotted an error about this and went through and fixed this in all of my repos the other day: https://github.com/search?o=desc&q=user%3Asimonw+google-github-actions%2Fsetup-gcloud%40v0&s=committer-date&type=Commits |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Pin setup-gcloud to v0 instead of master 1173828092 | |
1076638278 | https://github.com/simonw/datasette/issues/1670#issuecomment-1076638278 | https://api.github.com/repos/simonw/datasette/issues/1670 | IC_kwDOBm6k_c5ALDJG | simonw 9599 | 2022-03-23T17:50:55Z | 2022-03-23T17:50:55Z | OWNER | Release notes are mostly written for the alpha, just need to clean them up a bit https://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/docs/changelog.rst#061a0-2022-03-19 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ship Datasette 0.61 1174423568 | |
1075438684 | https://github.com/simonw/datasette/issues/1681#issuecomment-1075438684 | https://api.github.com/repos/simonw/datasette/issues/1681 | IC_kwDOBm6k_c5AGeRc | simonw 9599 | 2022-03-22T17:45:50Z | 2022-03-22T17:49:09Z | OWNER | I would expect this to break against SQL views that include calculated columns though - something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Potential bug in numeric handling where_clause for filters 1177101697 | |
1075437598 | https://github.com/simonw/datasette/issues/1681#issuecomment-1075437598 | https://api.github.com/repos/simonw/datasette/issues/1681 | IC_kwDOBm6k_c5AGeAe | simonw 9599 | 2022-03-22T17:44:42Z | 2022-03-22T17:45:04Z | OWNER | My hunch is that this mechanism doesn't actually do anything useful at all, because of the type conversion that automatically happens for data from tables based on the column type affinities, see: - #1671 So either remove the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Potential bug in numeric handling where_clause for filters 1177101697 | |
1075432283 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGctb | simonw 9599 | 2022-03-22T17:39:04Z | 2022-03-22T17:43:12Z | OWNER | Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like Though... it looks like there's a bug in that? It doesn't account for |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1075435185 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075435185 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGdax | simonw 9599 | 2022-03-22T17:42:09Z | 2022-03-22T17:42:09Z | OWNER | Also made me realize that this query:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1075428030 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075428030 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGbq- | simonw 9599 | 2022-03-22T17:34:30Z | 2022-03-22T17:34:30Z | OWNER | No, I think I need to use Even the details from the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1075425513 | https://github.com/simonw/datasette/issues/1671#issuecomment-1075425513 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5AGbDp | simonw 9599 | 2022-03-22T17:31:53Z | 2022-03-22T17:31:53Z | OWNER | The alternative to using This feels a bit neater to me, but I still then need to solve the problem of how to identify the "type" of a column that I want to use in a query. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074479932 | https://github.com/simonw/datasette/issues/339#issuecomment-1074479932 | https://api.github.com/repos/simonw/datasette/issues/339 | IC_kwDOBm6k_c5AC0M8 | simonw 9599 | 2022-03-21T22:22:34Z | 2022-03-21T22:22:34Z | OWNER | Closing this as obsolete since Datasette no longer uses Sanic. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose SANIC_RESPONSE_TIMEOUT config option in a sensible way 340396247 | |
1074479768 | https://github.com/simonw/datasette/issues/276#issuecomment-1074479768 | https://api.github.com/repos/simonw/datasette/issues/276 | IC_kwDOBm6k_c5AC0KY | simonw 9599 | 2022-03-21T22:22:20Z | 2022-03-21T22:22:20Z | OWNER | I'm closing this issue because this is now solved by a number of neat plugins:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Handle spatialite geometry columns better 324835838 | |
1074478299 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074478299 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACzzb | simonw 9599 | 2022-03-21T22:20:26Z | 2022-03-21T22:20:26Z | OWNER | Thinking about options for fixing this... The following query works fine:
If someone clicks on "View and edit SQL" from a filtered table page I don't want them to have to wonder why that But... for querying views, the So one fix would be to get the SQL generating logic to use casts like this any time it is operating against a view. An even better fix would be to detect which columns in a view come from a table and which ones might not, and only use casts for the columns that aren't definitely from a table. The trick I was exploring here might be able to help with that: - #1293 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074470568 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074470568 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACx6o | simonw 9599 | 2022-03-21T22:11:14Z | 2022-03-21T22:12:49Z | OWNER | I wonder if this will be a problem with generated columns, or with SQLite strict tables? My hunch is that strict tables will continue to work without any changes, because https://www.sqlite.org/stricttables.html says nothing about their impact on comparison operations. I should test this to make absolutely sure though. Generated columns have a type, so my hunch is they will continue to work fine too. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074468450 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074468450 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACxZi | simonw 9599 | 2022-03-21T22:08:35Z | 2022-03-21T22:10:00Z | OWNER | Relevant section of the SQLite documentation: 3.2. Affinity Of Expressions:
In your example, Then 4.2. Type Conversions Prior To Comparison fills in the rest:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074465536 | https://github.com/simonw/datasette/issues/1671#issuecomment-1074465536 | https://api.github.com/repos/simonw/datasette/issues/1671 | IC_kwDOBm6k_c5ACwsA | simonw 9599 | 2022-03-21T22:04:31Z | 2022-03-21T22:04:31Z | OWNER | Oh this is fascinating! I replicated the bug (thanks for the steps to reproduce) and it looks like this is down to the following: Against views, This doesn't happen against tables because of SQLite's type affinity mechanism, which handles the type conversion automatically. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Filters fail to work correctly against calculated numeric columns returned by SQL views because type affinity rules do not apply 1174655187 | |
1074459746 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074459746 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACvRi | simonw 9599 | 2022-03-21T21:55:45Z | 2022-03-21T21:55:45Z | OWNER | I'm going to change the original logic to set n=1 for times that are |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074458506 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074458506 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACu-K | simonw 9599 | 2022-03-21T21:53:47Z | 2022-03-21T21:53:47Z | OWNER | Oh interesting, it turns out there is ONE place in the code that sets the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074454687 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074454687 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACuCf | simonw 9599 | 2022-03-21T21:48:02Z | 2022-03-21T21:48:02Z | OWNER | Here's another microbenchmark that measures how many nanoseconds it takes to run 1,000 vmops: ```python import sqlite3 import time db = sqlite3.connect(":memory:") i = 0 out = [] def count(): global i i += 1000 out.append(((i, time.perf_counter_ns()))) db.set_progress_handler(count, 1000) print("Start:", time.perf_counter_ns()) all = db.execute(""" with recursive counter(x) as ( select 0 union select x + 1 from counter ) select * from counter limit 10000; """).fetchall() print("End:", time.perf_counter_ns()) print() print("So how long does it take to execute 1000 ops?") prev_time_ns = None
for i, time_ns in out:
if prev_time_ns is not None:
print(time_ns - prev_time_ns, "ns")
prev_time_ns = time_ns
So how long does it take to execute 1000 ops? 47290 ns 49573 ns 48226 ns 45674 ns 53238 ns 47313 ns 52346 ns 48689 ns 47092 ns 87596 ns 69999 ns 52522 ns 52809 ns 53259 ns 52478 ns 53478 ns 65812 ns ``` 87596ns is 0.087596ms - so even a measure rate of every 1000 ops is easily finely grained enough to capture differences of less than 0.1ms. If anything I could bump that default 1000 up - and I can definitely eliminate the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074446576 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074446576 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACsDw | simonw 9599 | 2022-03-21T21:38:27Z | 2022-03-21T21:38:27Z | OWNER | OK here's a microbenchmark script: ```python import sqlite3 import timeit db = sqlite3.connect(":memory:") db_with_progress_handler_1 = sqlite3.connect(":memory:") db_with_progress_handler_1000 = sqlite3.connect(":memory:") db_with_progress_handler_1.set_progress_handler(lambda: None, 1) db_with_progress_handler_1000.set_progress_handler(lambda: None, 1000) def execute_query(db): cursor = db.execute(""" with recursive counter(x) as ( select 0 union select x + 1 from counter ) select * from counter limit 10000; """) list(cursor.fetchall()) print("Without progress_handler") print(timeit.timeit(lambda: execute_query(db), number=100)) print("progress_handler every 1000 ops") print(timeit.timeit(lambda: execute_query(db_with_progress_handler_1000), number=100)) print("progress_handler every 1 op")
print(timeit.timeit(lambda: execute_query(db_with_progress_handler_1), number=100))
So running every 1000 ops makes almost no difference at all, but running every single op is a 3.2x performance degradation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074439309 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074439309 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACqSN | simonw 9599 | 2022-03-21T21:28:58Z | 2022-03-21T21:28:58Z | OWNER | David Raymond solved it there: https://sqlite.org/forum/forumpost/330c8532d8a88bcd
Sure enough, adding that gets the VM steps number up to 190,007 which is close enough that I'm happy. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074378472 | https://github.com/simonw/datasette/issues/1676#issuecomment-1074378472 | https://api.github.com/repos/simonw/datasette/issues/1676 | IC_kwDOBm6k_c5ACbbo | simonw 9599 | 2022-03-21T20:18:10Z | 2022-03-21T20:18:10Z | OWNER | Maybe there is a better name for this method that helps emphasize its cascading nature. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Reconsider ensure_permissions() logic, can it be less confusing? 1175690070 | |
1074347023 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074347023 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACTwP | simonw 9599 | 2022-03-21T19:48:59Z | 2022-03-21T19:48:59Z | OWNER | Posed a question about that here: https://sqlite.org/forum/forumpost/de9ff10fa7 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074341924 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074341924 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACSgk | simonw 9599 | 2022-03-21T19:42:08Z | 2022-03-21T19:42:08Z | OWNER | Here's the Python-C implementation of It calls https://www.sqlite.org/c3ref/progress_handler.html says:
So maybe VM-steps and virtual machine instructions are different things? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074337997 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074337997 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACRjN | simonw 9599 | 2022-03-21T19:37:08Z | 2022-03-21T19:37:08Z | OWNER | This is weird: ```python import sqlite3 db = sqlite3.connect(":memory:") i = 0 def count(): global i i += 1 db.set_progress_handler(count, 1) db.execute(""" with recursive counter(x) as ( select 0 union select x + 1 from counter ) select * from counter limit 10000; """) print(i)
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074332718 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074332718 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACQQu | simonw 9599 | 2022-03-21T19:31:10Z | 2022-03-21T19:31:10Z | OWNER | How long does it take for SQLite to execute 1000 opcodes anyway? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074332325 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074332325 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACQKl | simonw 9599 | 2022-03-21T19:30:44Z | 2022-03-21T19:30:44Z | OWNER | So it looks like even for facet suggestion |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074331743 | https://github.com/simonw/datasette/issues/1679#issuecomment-1074331743 | https://api.github.com/repos/simonw/datasette/issues/1679 | IC_kwDOBm6k_c5ACQBf | simonw 9599 | 2022-03-21T19:30:05Z | 2022-03-21T19:30:05Z | OWNER | https://github.com/simonw/datasette/blob/1a7750eb29fd15dd2eea3b9f6e33028ce441b143/datasette/app.py#L118-L122 sets it to 50ms for facet suggestion but that's not going to pass
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: how much overhead does the n=1 time limit have? 1175854982 | |
1074321862 | https://github.com/simonw/datasette/issues/1660#issuecomment-1074321862 | https://api.github.com/repos/simonw/datasette/issues/1660 | IC_kwDOBm6k_c5ACNnG | simonw 9599 | 2022-03-21T19:19:01Z | 2022-03-21T19:19:01Z | OWNER | I've simplified this a ton now. I'm going to keep working on this in the long-term but I think this issue can be closed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor and simplify Datasette routing and views 1170144879 | |
1074302559 | https://github.com/simonw/datasette/issues/1678#issuecomment-1074302559 | https://api.github.com/repos/simonw/datasette/issues/1678 | IC_kwDOBm6k_c5ACI5f | simonw 9599 | 2022-03-21T19:04:03Z | 2022-03-21T19:04:03Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make `check_visibility()` a documented API 1175715988 | ||
1074287177 | https://github.com/simonw/datasette/issues/1660#issuecomment-1074287177 | https://api.github.com/repos/simonw/datasette/issues/1660 | IC_kwDOBm6k_c5ACFJJ | simonw 9599 | 2022-03-21T18:51:42Z | 2022-03-21T18:51:42Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor and simplify Datasette routing and views 1170144879 | |
1074243540 | https://github.com/simonw/sqlite-utils/issues/417#issuecomment-1074243540 | https://api.github.com/repos/simonw/sqlite-utils/issues/417 | IC_kwDOCGYnMM5AB6fU | simonw 9599 | 2022-03-21T18:08:03Z | 2022-03-21T18:08:03Z | OWNER | I've not really thought about standards as much here as I should. It looks like there are two competing specs for newline-delimited JSON! http://ndjson.org/ is the one I've been using in
https://jsonlines.org/ is the other one. It is slightly less clear, but it does say this:
My interpretation of both of these is that newlines in the middle of a JSON object shouldn't be allowed. So what's The The thing I like about newline-delimited JSON is that it's really trivial to parse - loop through each line, run it through Unless someone has written a robust Python implementation of a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
insert fails on JSONL with whitespace 1175744654 | |
1074184240 | https://github.com/simonw/datasette/issues/1677#issuecomment-1074184240 | https://api.github.com/repos/simonw/datasette/issues/1677 | IC_kwDOBm6k_c5ABsAw | simonw 9599 | 2022-03-21T17:20:17Z | 2022-03-21T17:20:17Z | OWNER | This is weirdly different from how |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Remove `check_permission()` from `BaseView` 1175694248 | |
1074180312 | https://github.com/simonw/datasette/issues/1676#issuecomment-1074180312 | https://api.github.com/repos/simonw/datasette/issues/1676 | IC_kwDOBm6k_c5ABrDY | simonw 9599 | 2022-03-21T17:16:45Z | 2022-03-21T17:16:45Z | OWNER | When looking at this code earlier I assumed that the following would check each permission in turn and fail if any of them failed:
If that is indeed the right abstraction, I need to work to make the documentation as clear as possible. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Reconsider ensure_permissions() logic, can it be less confusing? 1175690070 | |
1074178865 | https://github.com/simonw/datasette/issues/1676#issuecomment-1074178865 | https://api.github.com/repos/simonw/datasette/issues/1676 | IC_kwDOBm6k_c5ABqsx | simonw 9599 | 2022-03-21T17:15:27Z | 2022-03-21T17:15:27Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Reconsider ensure_permissions() logic, can it be less confusing? 1175690070 | ||
1074177827 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074177827 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABqcj | simonw 9599 | 2022-03-21T17:14:31Z | 2022-03-21T17:14:31Z | OWNER | Updated documentation: https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/docs/internals.rst#await-ensure_permissionsactor-permissions
That's pretty hard to understand! I'm going to open a separate issue to reconsider if this is a useful enough abstraction given how confusing it is. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074161523 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074161523 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABmdz | simonw 9599 | 2022-03-21T16:59:55Z | 2022-03-21T17:00:03Z | OWNER | Also calling that function |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074158890 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074158890 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABl0q | simonw 9599 | 2022-03-21T16:57:15Z | 2022-03-21T16:57:15Z | OWNER | Idea: A new |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074156779 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074156779 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABlTr | simonw 9599 | 2022-03-21T16:55:08Z | 2022-03-21T16:56:02Z | OWNER | One benefit of the current design of I could return an object which evaluates to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074143209 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074143209 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABh_p | simonw 9599 | 2022-03-21T16:46:05Z | 2022-03-21T16:46:05Z | OWNER | The other difference though is that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074142617 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074142617 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABh2Z | simonw 9599 | 2022-03-21T16:45:27Z | 2022-03-21T16:45:27Z | OWNER | Though at that point So maybe
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074141457 | https://github.com/simonw/datasette/issues/1675#issuecomment-1074141457 | https://api.github.com/repos/simonw/datasette/issues/1675 | IC_kwDOBm6k_c5ABhkR | simonw 9599 | 2022-03-21T16:44:09Z | 2022-03-21T16:44:09Z | OWNER | A slightly odd thing about these methods is that they either fail silently or they raise a Maybe they should instead return |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Extract out `check_permissions()` from `BaseView 1175648453 | |
1074136176 | https://github.com/simonw/datasette/issues/1660#issuecomment-1074136176 | https://api.github.com/repos/simonw/datasette/issues/1660 | IC_kwDOBm6k_c5ABgRw | simonw 9599 | 2022-03-21T16:38:46Z | 2022-03-21T16:38:46Z | OWNER | I'm going to refactor this stuff out and document it so it can be easily used by plugins: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor and simplify Datasette routing and views 1170144879 | |
1074019047 | https://github.com/simonw/datasette/issues/526#issuecomment-1074019047 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5ABDrn | simonw 9599 | 2022-03-21T15:09:56Z | 2022-03-21T15:09:56Z | OWNER | I should research how much overhead creating a new connection costs - it may be that an easy way to solve this is to create A dedicated connection for the query and then close that connection at the end. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1074017633 | https://github.com/simonw/datasette/issues/1177#issuecomment-1074017633 | https://api.github.com/repos/simonw/datasette/issues/1177 | IC_kwDOBm6k_c5ABDVh | simonw 9599 | 2022-03-21T15:08:51Z | 2022-03-21T15:08:51Z | OWNER | Related: - #1062 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability to stream all rows as newline-delimited JSON 780153562 | |
1073468996 | https://github.com/simonw/sqlite-utils/issues/415#issuecomment-1073468996 | https://api.github.com/repos/simonw/sqlite-utils/issues/415 | IC_kwDOCGYnMM4_-9ZE | simonw 9599 | 2022-03-21T04:14:42Z | 2022-03-21T04:14:42Z | OWNER | I can fix this like so: ``` % sqlite-utils convert demo.db demo foo '{"foo": "bar"}' --multi --dry-run abc --- becomes: {"foo": "bar"} Would affect 1 row
|
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Convert with `--multi` and `--dry-run` flag does not work 1171599874 | |
1073463375 | https://github.com/simonw/sqlite-utils/issues/415#issuecomment-1073463375 | https://api.github.com/repos/simonw/sqlite-utils/issues/415 | IC_kwDOCGYnMM4_-8BP | simonw 9599 | 2022-03-21T04:02:36Z | 2022-03-21T04:02:36Z | OWNER | Thanks for the really clear steps to reproduce! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Convert with `--multi` and `--dry-run` flag does not work 1171599874 | |
1073456222 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073456222 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-6Re | simonw 9599 | 2022-03-21T03:45:52Z | 2022-03-21T03:45:52Z | OWNER | Needs tests and documentation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073456155 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073456155 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-6Qb | simonw 9599 | 2022-03-21T03:45:37Z | 2022-03-21T03:45:37Z | OWNER | Prototype: ```diff diff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py index 8255b56..0a3693e 100644 --- a/sqlite_utils/cli.py +++ b/sqlite_utils/cli.py @@ -2583,7 +2583,11 @@ def generate_convert_help(): """ ).strip() recipe_names = [ - n for n in dir(recipes) if not n.startswith("") and n not in ("json", "parser") + n + for n in dir(recipes) + if not n.startswith("_") + and n not in ("json", "parser") + and callable(getattr(recipes, n)) ] for name in recipe_names: fn = getattr(recipes, name) diff --git a/sqlite_utils/recipes.py b/sqlite_utils/recipes.py index 6918661..569c30d 100644 --- a/sqlite_utils/recipes.py +++ b/sqlite_utils/recipes.py @@ -1,17 +1,38 @@ from dateutil import parser import json +IGNORE = object() +SET_NULL = object() -def parsedate(value, dayfirst=False, yearfirst=False): + +def parsedate(value, dayfirst=False, yearfirst=False, errors=None): "Parse a date and convert it to ISO date format: yyyy-mm-dd" - return ( - parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).date().isoformat() - ) + try: + return ( + parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst) + .date() + .isoformat() + ) + except parser.ParserError: + if errors is IGNORE: + return value + elif errors is SET_NULL: + return None + else: + raise -def parsedatetime(value, dayfirst=False, yearfirst=False): +def parsedatetime(value, dayfirst=False, yearfirst=False, errors=None): "Parse a datetime and convert it to ISO datetime format: yyyy-mm-ddTHH:MM:SS" - return parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).isoformat() + try: + return parser.parse(value, dayfirst=dayfirst, yearfirst=yearfirst).isoformat() + except parser.ParserError: + if errors is IGNORE: + return value + elif errors is SET_NULL: + return None + else: + raise def jsonsplit(value, delimiter=",", type=str): ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073455905 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073455905 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-6Mh | simonw 9599 | 2022-03-21T03:44:47Z | 2022-03-21T03:45:00Z | OWNER | This is quite nice:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073453370 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073453370 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-5k6 | simonw 9599 | 2022-03-21T03:41:06Z | 2022-03-21T03:41:06Z | OWNER | I'm going to try the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073453230 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073453230 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-5iu | simonw 9599 | 2022-03-21T03:40:37Z | 2022-03-21T03:40:37Z | OWNER | I think the options here should be:
These need to be indicated by parameters to the Some design options:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073451659 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073451659 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-5KL | simonw 9599 | 2022-03-21T03:35:01Z | 2022-03-21T03:35:01Z | OWNER | I confirmed that if it fails for any value ALL values are left alone, since it runs in a transaction. Here's the code that does that: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073450588 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073450588 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-45c | simonw 9599 | 2022-03-21T03:32:58Z | 2022-03-21T03:32:58Z | OWNER | Then I ran this to convert
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073448904 | https://github.com/simonw/sqlite-utils/issues/416#issuecomment-1073448904 | https://api.github.com/repos/simonw/sqlite-utils/issues/416 | IC_kwDOCGYnMM4_-4fI | simonw 9599 | 2022-03-21T03:28:12Z | 2022-03-21T03:30:37Z | OWNER | Generating a test database using a pattern from https://www.geekytidbits.com/date-range-table-sqlite/
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Options for how `r.parsedate()` should handle invalid dates 1173023272 | |
1073366630 | https://github.com/simonw/datasette/issues/1510#issuecomment-1073366630 | https://api.github.com/repos/simonw/datasette/issues/1510 | IC_kwDOBm6k_c4_-kZm | simonw 9599 | 2022-03-20T22:59:33Z | 2022-03-20T22:59:33Z | OWNER | I really like the idea of making this effectively the same thing as the fully documented, stable JSON API that comes as part of 1.0. If you want to know what will be available to your templates, consult the API documentation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette 1.0 documented template context (maybe via API docs) 1054244712 | |
1073366436 | https://github.com/simonw/datasette/issues/1674#issuecomment-1073366436 | https://api.github.com/repos/simonw/datasette/issues/1674 | IC_kwDOBm6k_c4_-kWk | simonw 9599 | 2022-03-20T22:58:40Z | 2022-03-20T22:58:40Z | OWNER | This will probably happen as part of turning this into an officially documented API that serves the template context for the homepage: - #1510 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Tweak design of /.json 1174717287 | |
1073362979 | https://github.com/simonw/datasette/issues/1355#issuecomment-1073362979 | https://api.github.com/repos/simonw/datasette/issues/1355 | IC_kwDOBm6k_c4_-jgj | simonw 9599 | 2022-03-20T22:38:53Z | 2022-03-20T22:38:53Z | OWNER | Built a research prototype: ```diff diff --git a/datasette/app.py b/datasette/app.py index 5c8101a..5cd3e63 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -1,6 +1,7 @@ import asyncio import asgi_csrf import collections +import contextlib import datetime import functools import glob @@ -1490,3 +1491,11 @@ class DatasetteClient: return await client.request( method, self._fix(path, avoid_path_rewrites), kwargs ) + + @contextlib.asynccontextmanager + async def stream(self, method, path, kwargs): + async with httpx.AsyncClient(app=self.app) as client: + print("async with as client") + async with client.stream(method, self._fix(path), **kwargs) as response: + print("async with client.stream about to yield response") + yield response diff --git a/datasette/cli.py b/datasette/cli.py index 3c6e1b2..3025ead 100644 --- a/datasette/cli.py +++ b/datasette/cli.py @@ -585,11 +585,19 @@ def serve( asyncio.get_event_loop().run_until_complete(check_databases(ds))
No idea why. I dropped this for the moment. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette --get should efficiently handle streaming CSV 910088936 | |
1073361986 | https://github.com/simonw/datasette/issues/1673#issuecomment-1073361986 | https://api.github.com/repos/simonw/datasette/issues/1673 | IC_kwDOBm6k_c4_-jRC | simonw 9599 | 2022-03-20T22:31:41Z | 2022-03-20T22:34:06Z | OWNER | Maybe it's because Actually no, I'm caching that already: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Streaming CSV spends a lot of time in `table_column_details` 1174708375 | |
1073355818 | https://github.com/simonw/datasette/issues/1672#issuecomment-1073355818 | https://api.github.com/repos/simonw/datasette/issues/1672 | IC_kwDOBm6k_c4_-hwq | simonw 9599 | 2022-03-20T21:52:38Z | 2022-03-20T21:52:38Z | OWNER | That means taking on these issues: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor CSV handling code out of DataView 1174697144 | |
1073355032 | https://github.com/simonw/datasette/issues/1660#issuecomment-1073355032 | https://api.github.com/repos/simonw/datasette/issues/1660 | IC_kwDOBm6k_c4_-hkY | simonw 9599 | 2022-03-20T21:46:43Z | 2022-03-20T21:46:43Z | OWNER | I think the way to get rid of most of the remaining complexity in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor and simplify Datasette routing and views 1170144879 | |
1073330388 | https://github.com/simonw/sqlite-utils/issues/140#issuecomment-1073330388 | https://api.github.com/repos/simonw/sqlite-utils/issues/140 | IC_kwDOCGYnMM4_-bjU | simonw 9599 | 2022-03-20T19:44:39Z | 2022-03-20T19:45:45Z | OWNER | Alternative idea for specifying types: accept a Python expression, then use Python type literal syntax. For example:
Where |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Idea: insert-files mechanism for adding extra columns with fixed values 688351054 | |
1073143413 | https://github.com/simonw/datasette/issues/1669#issuecomment-1073143413 | https://api.github.com/repos/simonw/datasette/issues/1669 | IC_kwDOBm6k_c4_9t51 | simonw 9599 | 2022-03-20T01:24:36Z | 2022-03-20T01:24:36Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Release 0.61 alpha 1174404647 | ||
1073137170 | https://github.com/simonw/datasette/issues/1669#issuecomment-1073137170 | https://api.github.com/repos/simonw/datasette/issues/1669 | IC_kwDOBm6k_c4_9sYS | simonw 9599 | 2022-03-20T00:35:52Z | 2022-03-20T00:35:52Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Release 0.61 alpha 1174404647 | ||
1073136896 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9sUA | simonw 9599 | 2022-03-20T00:33:23Z | 2022-03-20T00:33:23Z | OWNER | I'm going to release this as a 0.61 alpha so I can more easily depend on it from |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073136686 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073136686 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9sQu | simonw 9599 | 2022-03-20T00:31:13Z | 2022-03-20T00:31:13Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | ||
1073135433 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073135433 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9r9J | simonw 9599 | 2022-03-20T00:20:36Z | 2022-03-20T00:20:36Z | OWNER | Building this plugin instantly revealed that all of the links - on the homepage and the database page and so on - are incorrect: ```python from datasette import hookimpl @hookimpl def startup(datasette): db = datasette.get_database("fixtures2") db.route = "alternative-route" ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073134816 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073134816 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9rzg | simonw 9599 | 2022-03-20T00:16:22Z | 2022-03-20T00:16:22Z | OWNER | I'm going to add a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073134206 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073134206 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9rp- | simonw 9599 | 2022-03-20T00:12:03Z | 2022-03-20T00:12:03Z | OWNER | I'd like to have a live demo of this up on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073126264 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073126264 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9pt4 | simonw 9599 | 2022-03-19T22:59:30Z | 2022-03-19T22:59:30Z | OWNER | Also need to update the Need to add tests that check the links in the HTML and can confirm this is working correctly. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073125334 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073125334 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9pfW | simonw 9599 | 2022-03-19T22:53:55Z | 2022-03-19T22:53:55Z | OWNER | Need to update documentation in a few places - e.g. https://docs.datasette.io/en/stable/internals.html#remove-database-name
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073112104 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073112104 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9mQo | simonw 9599 | 2022-03-19T21:08:21Z | 2022-03-19T21:08:21Z | OWNER | I think I've got this working but I need to write a test for it that covers the rare case when the route is not the same thing as the database name. I'll do that with a new test. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073097394 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073097394 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9iqy | simonw 9599 | 2022-03-19T20:56:35Z | 2022-03-19T20:56:35Z | OWNER | I'm trying to think if there's any reason not to use Decision made: I'm going with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073076624 | https://github.com/simonw/datasette/issues/1667#issuecomment-1073076624 | https://api.github.com/repos/simonw/datasette/issues/1667 | IC_kwDOBm6k_c4_9dmQ | simonw 9599 | 2022-03-19T20:31:44Z | 2022-03-19T20:31:44Z | OWNER | I can now read |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make route matched pattern groups more consistent 1174302994 | |
1073076187 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073076187 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9dfb | simonw 9599 | 2022-03-19T20:28:20Z | 2022-03-19T20:28:20Z | OWNER | I'm going to keep |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073076136 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073076136 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9deo | simonw 9599 | 2022-03-19T20:27:44Z | 2022-03-19T20:27:44Z | OWNER | Pretty sure changing it will break some existing plugins though, including likely Datasette Desktop. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073076110 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073076110 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9deO | simonw 9599 | 2022-03-19T20:27:22Z | 2022-03-19T20:27:22Z | OWNER | The docs do currently describe Good thing I'm not at 1.0 yet so I can change that! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073076015 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073076015 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9dcv | simonw 9599 | 2022-03-19T20:26:32Z | 2022-03-19T20:26:32Z | OWNER | I'm inclined to redefine |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073075913 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073075913 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9dbJ | simonw 9599 | 2022-03-19T20:25:46Z | 2022-03-19T20:26:08Z | OWNER | The output of
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073075697 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073075697 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9dXx | simonw 9599 | 2022-03-19T20:24:06Z | 2022-03-19T20:24:06Z | OWNER | Right now if a database has a
But the output on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073073599 | https://github.com/simonw/datasette/issues/1660#issuecomment-1073073599 | https://api.github.com/repos/simonw/datasette/issues/1660 | IC_kwDOBm6k_c4_9c2_ | simonw 9599 | 2022-03-19T20:06:40Z | 2022-03-19T20:06:40Z | OWNER | This blocks: - #1668 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor and simplify Datasette routing and views 1170144879 | |
1073073579 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073073579 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9c2r | simonw 9599 | 2022-03-19T20:06:27Z | 2022-03-19T20:06:27Z | OWNER | Marking this as blocked until #1660 is done. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073073547 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073073547 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9c2L | simonw 9599 | 2022-03-19T20:06:07Z | 2022-03-19T20:06:07Z | OWNER | Implementing this is a little tricky because there's a whole lot of code that expects the The All the more reason to get rid of that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073043433 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073043433 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9Vfp | simonw 9599 | 2022-03-19T16:54:55Z | 2022-03-19T20:01:19Z | OWNER | Options:
- I like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073043713 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073043713 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9VkB | simonw 9599 | 2022-03-19T16:56:19Z | 2022-03-19T16:56:19Z | OWNER | Worth noting that the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073043350 | https://github.com/simonw/datasette/issues/1668#issuecomment-1073043350 | https://api.github.com/repos/simonw/datasette/issues/1668 | IC_kwDOBm6k_c4_9VeW | simonw 9599 | 2022-03-19T16:54:26Z | 2022-03-19T16:54:26Z | OWNER | The So need a different name for the path-that-is-used-in-the-URL. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Introduce concept of a database `route`, separate from its name 1174306154 | |
1073042554 | https://github.com/simonw/datasette/issues/1667#issuecomment-1073042554 | https://api.github.com/repos/simonw/datasette/issues/1667 | IC_kwDOBm6k_c4_9VR6 | simonw 9599 | 2022-03-19T16:50:01Z | 2022-03-19T16:52:35Z | OWNER | OK, I've made this more consistent - I still need to address the fact that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make route matched pattern groups more consistent 1174302994 | |
1073040072 | https://github.com/simonw/datasette/issues/1667#issuecomment-1073040072 | https://api.github.com/repos/simonw/datasette/issues/1667 | IC_kwDOBm6k_c4_9UrI | simonw 9599 | 2022-03-19T16:34:02Z | 2022-03-19T16:34:02Z | OWNER | I called it I think I'm going to go with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Make route matched pattern groups more consistent 1174302994 | |
1073039670 | https://github.com/simonw/datasette/issues/1666#issuecomment-1073039670 | https://api.github.com/repos/simonw/datasette/issues/1666 | IC_kwDOBm6k_c4_9Uk2 | simonw 9599 | 2022-03-19T16:31:08Z | 2022-03-19T16:31:57Z | OWNER | This does make it more interesting - it also highlights how inconsistent the way the capturing works is. Especially |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor URL routing to enable testing 1174162781 | |
1073039241 | https://github.com/simonw/datasette/issues/1666#issuecomment-1073039241 | https://api.github.com/repos/simonw/datasette/issues/1666 | IC_kwDOBm6k_c4_9UeJ | simonw 9599 | 2022-03-19T16:28:15Z | 2022-03-19T16:28:15Z | OWNER | This is more interesting if it also asserts against the captured matches from the pattern. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor URL routing to enable testing 1174162781 | |
1073037939 | https://github.com/simonw/datasette/issues/878#issuecomment-1073037939 | https://api.github.com/repos/simonw/datasette/issues/878 | IC_kwDOBm6k_c4_9UJz | simonw 9599 | 2022-03-19T16:19:30Z | 2022-03-19T16:19:30Z | OWNER | On revisiting https://gist.github.com/simonw/281eac9c73b062c3469607ad86470eb2 a few months later I'm having second thoughts about using But I still like the pattern as a way to resolve more complex cases like "to generate GeoJSON of the expanded view with labels, the label expansion code needs to run once at some before the GeoJSON formatting code does". So I'm going to stick with it a tiny bit longer, but maybe try to make it a lot more explicit when it's going to happen rather than having the main view methods themselves also use async DI. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
New pattern for views that return either JSON or HTML, available for plugins 648435885 | |
1072939780 | https://github.com/simonw/datasette/issues/1561#issuecomment-1072939780 | https://api.github.com/repos/simonw/datasette/issues/1561 | IC_kwDOBm6k_c4_88ME | simonw 9599 | 2022-03-19T04:45:40Z | 2022-03-19T04:45:40Z | OWNER | I ended up moving hashed URL mode out to a plugin in: - #647 If you're still interested in using it with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
add hash id to "_memory" url if hashed url mode is turned on and crossdb is also turned on 1082765654 | |
1072933875 | https://github.com/simonw/datasette/issues/1666#issuecomment-1072933875 | https://api.github.com/repos/simonw/datasette/issues/1666 | IC_kwDOBm6k_c4_86vz | simonw 9599 | 2022-03-19T04:03:42Z | 2022-03-19T04:03:42Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor URL routing to enable testing 1174162781 | ||
1072915936 | https://github.com/simonw/datasette/issues/1228#issuecomment-1072915936 | https://api.github.com/repos/simonw/datasette/issues/1228 | IC_kwDOBm6k_c4_82Xg | simonw 9599 | 2022-03-19T01:50:27Z | 2022-03-19T01:50:27Z | OWNER | Demo: https://latest.datasette.io/fixtures/facetable - which now has a column called |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error caused by faceting if a column called `n` exists 810397025 | |
1072908029 | https://github.com/simonw/datasette/issues/1228#issuecomment-1072908029 | https://api.github.com/repos/simonw/datasette/issues/1228 | IC_kwDOBm6k_c4_80b9 | simonw 9599 | 2022-03-19T00:57:54Z | 2022-03-19T00:57:54Z | OWNER | Yes! That's the problem. I was able to replicate it like so:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error caused by faceting if a column called `n` exists 810397025 | |
1072907680 | https://github.com/simonw/datasette/issues/1228#issuecomment-1072907680 | https://api.github.com/repos/simonw/datasette/issues/1228 | IC_kwDOBm6k_c4_80Wg | simonw 9599 | 2022-03-19T00:55:48Z | 2022-03-19T00:55:48Z | OWNER | ... unless your data had a column called |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error caused by faceting if a column called `n` exists 810397025 | |
1072907610 | https://github.com/simonw/datasette/issues/1228#issuecomment-1072907610 | https://api.github.com/repos/simonw/datasette/issues/1228 | IC_kwDOBm6k_c4_80Va | simonw 9599 | 2022-03-19T00:55:29Z | 2022-03-19T00:55:29Z | OWNER | It looks to me like something is causing the faceting query here to return a string when it was expected to return a number: I can't think of any way that a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
500 error caused by faceting if a column called `n` exists 810397025 | |
1072907200 | https://github.com/simonw/datasette/issues/1605#issuecomment-1072907200 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c4_80PA | simonw 9599 | 2022-03-19T00:52:54Z | 2022-03-19T00:53:45Z | OWNER | Had a thought about the implementation of this: it could make a really neat plugin. Something like
And the command would then:
All of that HTML parsing may be over-complicating things. It could alternatively accept options for which pages you want to export:
Or a really wild option: it could allow you to define the paths you want to export using a SQL query:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1072905467 | https://github.com/simonw/datasette/issues/1662#issuecomment-1072905467 | https://api.github.com/repos/simonw/datasette/issues/1662 | IC_kwDOBm6k_c4_8zz7 | simonw 9599 | 2022-03-19T00:42:23Z | 2022-03-19T00:42:23Z | OWNER | Those client-side SQLite tricks are really neat.
If you're not going to run any server-side code at all you don't need Datasette for this - you can upload the SQLite database file to any static hosting with CORS headers and load it into the client that way. In terms of static publishing, I do think there's something interesting about using Datasette to generate static sites. There's an issue discussing options for that over here:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[feature request] Publish to fully static website 1170497629 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
issue >1000