html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853567861,https://api.github.com/repos/simonw/sqlite-utils/issues/264,853567861,MDEyOklzc3VlQ29tbWVudDg1MzU2Nzg2MQ==,9599,2021-06-03T05:12:21Z,2021-06-03T05:12:21Z,OWNER,I think this is more likely to happen in Datasette than in sqlite-utils - see https://github.com/simonw/datasette/issues/1356 for thoughts on this.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907642546,
https://github.com/simonw/datasette/issues/526#issuecomment-853567413,https://api.github.com/repos/simonw/datasette/issues/526,853567413,MDEyOklzc3VlQ29tbWVudDg1MzU2NzQxMw==,9599,2021-06-03T05:11:27Z,2021-06-03T05:11:27Z,OWNER,"Another potential way to implement this would be to hold the SQLite connection open and execute the full query there.
I've avoided this in the past due to concerns of resource exhaustion - if multiple requests attempt this at the same time all of the connections in the pool will become tied up and the site will be unable to respond to further requests.
But... now that Datasette has authentication there's the possibility of making this feature only available to specific authenticated users - the `--root` user for example. Which avoids the danger while unlocking a super-useful feature.
Not to mention people who are running Datasette privately on their own laptop, or the proposed `--query` CLI feature in #1356.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459882902,
https://github.com/simonw/datasette/issues/1356#issuecomment-853566337,https://api.github.com/repos/simonw/datasette/issues/1356,853566337,MDEyOklzc3VlQ29tbWVudDg1MzU2NjMzNw==,9599,2021-06-03T05:08:32Z,2021-06-03T05:08:32Z,OWNER,Also relevant: CSV streaming for canned queries in #526 - even better if we could stream ANY SQL query.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1062#issuecomment-853566011,https://api.github.com/repos/simonw/datasette/issues/1062,853566011,MDEyOklzc3VlQ29tbWVudDg1MzU2NjAxMQ==,9599,2021-06-03T05:07:42Z,2021-06-03T05:07:42Z,OWNER,Implementing this would make #1356 a whole lot more interesting.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",732674148,
https://github.com/simonw/datasette/issues/1356#issuecomment-853565850,https://api.github.com/repos/simonw/datasette/issues/1356,853565850,MDEyOklzc3VlQ29tbWVudDg1MzU2NTg1MA==,9599,2021-06-03T05:07:21Z,2021-06-03T05:07:21Z,OWNER,"A problem with this is that if you're using `--query` you likely want ALL of the results - at the moment the only Datasette output type that can stream everything is `.csv` and plugin formats can't handle full streams, see #1062 and #1177.
So there's not much point implementing this unless we first make plugins able to add custom streaming formats.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853562891,https://api.github.com/repos/simonw/datasette/issues/1356,853562891,MDEyOklzc3VlQ29tbWVudDg1MzU2Mjg5MQ==,9599,2021-06-03T04:59:40Z,2021-06-03T04:59:40Z,OWNER,"It's weird that `--get` is documented on this page right now: https://docs.datasette.io/en/stable/getting_started.html#datasette-get
If I implement this I should build a new ""Datasette on the command-line"" documentation page to cover both `--get` and `--query`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853560870,https://api.github.com/repos/simonw/datasette/issues/1356,853560870,MDEyOklzc3VlQ29tbWVudDg1MzU2MDg3MA==,9599,2021-06-03T04:53:47Z,2021-06-03T04:53:56Z,OWNER,"This is also interesting when used in conjunction with the proposed `datasette insert` command from #1163 - Datasette would become a plugin-driven CLI tool for both ingesting and outputting data, as a side-effect of its web features.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853560389,https://api.github.com/repos/simonw/datasette/issues/1356,853560389,MDEyOklzc3VlQ29tbWVudDg1MzU2MDM4OQ==,9599,2021-06-03T04:52:13Z,2021-06-03T04:52:13Z,OWNER,I should implement #1355 for more efficient `--csv` streaming as part of this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853560237,https://api.github.com/repos/simonw/datasette/issues/1356,853560237,MDEyOklzc3VlQ29tbWVudDg1MzU2MDIzNw==,9599,2021-06-03T04:51:49Z,2021-06-03T04:51:49Z,OWNER,This feels like a relatively simple feature to implement that unlocks a whole new set of possible uses for Datasette - as described by @eyeseast in https://github.com/simonw/sqlite-utils/issues/264#issue-907642546.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/datasette/issues/1356#issuecomment-853559915,https://api.github.com/repos/simonw/datasette/issues/1356,853559915,MDEyOklzc3VlQ29tbWVudDg1MzU1OTkxNQ==,9599,2021-06-03T04:50:52Z,2021-06-03T04:50:52Z,OWNER,"What happens if you pass multiple databases? The `--query` would be executed against the first one. And if you pass `--crossdb` it would be executed against the `/_memory` database and would support cross-database joins.
Key thing here is that output plugins are supported (also plugins that add new SQL functions), making many Datasette plugins usable from the command-line.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910092577,
https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853558741,https://api.github.com/repos/simonw/sqlite-utils/issues/264,853558741,MDEyOklzc3VlQ29tbWVudDg1MzU1ODc0MQ==,9599,2021-06-03T04:47:19Z,2021-06-03T04:47:19Z,OWNER,"This inspired me to re-examine how `--get` works, hence this issue: https://github.com/simonw/datasette/issues/1355","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907642546,
https://github.com/simonw/datasette/issues/1355#issuecomment-853557439,https://api.github.com/repos/simonw/datasette/issues/1355,853557439,MDEyOklzc3VlQ29tbWVudDg1MzU1NzQzOQ==,9599,2021-06-03T04:43:14Z,2021-06-03T04:43:14Z,OWNER,"It's using `TestClient` at the moment which is a wrapper around `httpx` (as of ) that uses the `@async_to_sync` decorator to hide the async nature.
https://github.com/simonw/datasette/blob/f78ebdc04537a6102316d6dbbf6c887565806078/datasette/utils/testing.py#L102-L156
Maybe the fix here is to switch the `--get` implementation to using `httpx` directly with https://www.python-httpx.org/async/#streaming-responses","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",910088936,
https://github.com/simonw/sqlite-utils/issues/264#issuecomment-853553754,https://api.github.com/repos/simonw/sqlite-utils/issues/264,853553754,MDEyOklzc3VlQ29tbWVudDg1MzU1Mzc1NA==,9599,2021-06-03T04:32:42Z,2021-06-03T04:36:36Z,OWNER,"This is a really interesting thought. I've so far resisted the temptation to add plugins to `sqlite-utils`, partly to avoid overlap with Datasette - but I'm open to discussing it.
There's actually a way for you to do what you're describing using `datasette` on the command-line, though it's a little obscure - also Datasette doesn't yet have a GeoJSON output extension, though it really should have one.
Here's an example using [datasette-yaml](https://datasette.io/plugins/datasette-yaml):
```
datasette /tmp/covid.db --get='/covid/ny_times_us_counties.yaml'
- rowid: 1
date: '2020-01-21'
county: Snohomish
state: Washington
fips: 53061
cases: 1
deaths: 0
- rowid: 2
date: '2020-01-22'
county: Snohomish
state: Washington
fips: 53061
cases: 1
deaths: 0
```
It even works with arbitrary SQL queries, though you might have to apply URL encoding to the `--get` string (this seems to work though):
```
datasette /tmp/covid.db --get='/covid.yaml?sql=select * from ny_times_us_counties limit 2'
- date: '2020-01-21'
county: Snohomish
state: Washington
fips: 53061
cases: 1
deaths: 0
- date: '2020-01-22'
county: Snohomish
state: Washington
fips: 53061
cases: 1
deaths: 0
```
Here's the documentation for `--get`: https://docs.datasette.io/en/latest/getting_started.html#datasette-get","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",907642546,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853554550,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853554550,MDEyOklzc3VlQ29tbWVudDg1MzU1NDU1MA==,9599,2021-06-03T04:34:38Z,2021-06-03T04:34:38Z,OWNER,Documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#listing-indexes,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853548442,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853548442,MDEyOklzc3VlQ29tbWVudDg1MzU0ODQ0Mg==,9599,2021-06-03T04:16:49Z,2021-06-03T04:16:49Z,OWNER,Needs to show the table each index applies to.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853547681,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853547681,MDEyOklzc3VlQ29tbWVudDg1MzU0NzY4MQ==,9599,2021-06-03T04:14:28Z,2021-06-03T04:14:28Z,OWNER,"This looks good:
```
% sqlite-utils indexes /tmp/covid.db -t
index_name seqno cid name desc coll key
------------------------------------------------------ ------- ----- ----------------- ------ ------ -----
idx_johns_hopkins_csse_daily_reports_combined_key 0 12 combined_key 0 BINARY 1
idx_johns_hopkins_csse_daily_reports_country_or_region 0 1 country_or_region 0 BINARY 1
idx_johns_hopkins_csse_daily_reports_province_or_state 0 2 province_or_state 0 BINARY 1
idx_johns_hopkins_csse_daily_reports_day 0 0 day 0 BINARY 1
idx_ny_times_us_counties_date 0 0 date 1 BINARY 1
idx_ny_times_us_counties_fips 0 3 fips 0 BINARY 1
idx_ny_times_us_counties_county 0 1 county 0 BINARY 1
idx_ny_times_us_counties_state 0 2 state 0 BINARY 1
% sqlite-utils indexes /tmp/covid.db -t --aux
index_name seqno cid name desc coll key
------------------------------------------------------ ------- ----- ----------------- ------ ------ -----
idx_johns_hopkins_csse_daily_reports_combined_key 0 12 combined_key 0 BINARY 1
idx_johns_hopkins_csse_daily_reports_combined_key 1 -1 0 BINARY 0
idx_johns_hopkins_csse_daily_reports_country_or_region 0 1 country_or_region 0 BINARY 1
idx_johns_hopkins_csse_daily_reports_country_or_region 1 -1 0 BINARY 0
idx_johns_hopkins_csse_daily_reports_province_or_state 0 2 province_or_state 0 BINARY 1
idx_johns_hopkins_csse_daily_reports_province_or_state 1 -1 0 BINARY 0
idx_johns_hopkins_csse_daily_reports_day 0 0 day 0 BINARY 1
idx_johns_hopkins_csse_daily_reports_day 1 -1 0 BINARY 0
idx_ny_times_us_counties_date 0 0 date 1 BINARY 1
idx_ny_times_us_counties_date 1 -1 0 BINARY 0
idx_ny_times_us_counties_fips 0 3 fips 0 BINARY 1
idx_ny_times_us_counties_fips 1 -1 0 BINARY 0
idx_ny_times_us_counties_county 0 1 county 0 BINARY 1
idx_ny_times_us_counties_county 1 -1 0 BINARY 0
idx_ny_times_us_counties_state 0 2 state 0 BINARY 1
idx_ny_times_us_counties_state 1 -1 0 BINARY 0
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853546818,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853546818,MDEyOklzc3VlQ29tbWVudDg1MzU0NjgxOA==,9599,2021-06-03T04:11:46Z,2021-06-03T04:11:46Z,OWNER,"By default I won't return auxiliary columns, but I'll offer a `--aux` option to return them.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853545743,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853545743,MDEyOklzc3VlQ29tbWVudDg1MzU0NTc0Mw==,9599,2021-06-03T04:08:04Z,2021-06-03T04:08:04Z,OWNER,"Figuring out the right queries:
https://covid-19.datasettes.com/covid?sql=select+sqlite_master.name%2C+i.*+from+sqlite_master%0D%0Ajoin+pragma_index_list%28sqlite_master.name%29+i%0D%0Awhere+type+%3D+%27table%27
This query shows all columns across all indexes across all tables:
```sql
select i.name as index_name, xinfo.* from sqlite_master
join pragma_index_list(sqlite_master.name) i
join pragma_index_xinfo(index_name) xinfo
where sqlite_master.type = 'table'
```
https://covid-19.datasettes.com/covid?sql=select+i.name+as+index_name%2C+xinfo.*+from+sqlite_master%0D%0Ajoin+pragma_index_list%28sqlite_master.name%29+i%0D%0Ajoin+pragma_index_xinfo%28index_name%29+xinfo%0D%0Awhere+sqlite_master.type+%3D+%27table%27","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/263#issuecomment-853544493,https://api.github.com/repos/simonw/sqlite-utils/issues/263,853544493,MDEyOklzc3VlQ29tbWVudDg1MzU0NDQ5Mw==,9599,2021-06-03T04:03:59Z,2021-06-03T04:03:59Z,OWNER,"Here's how `sqlite-utils triggers` works: https://github.com/simonw/sqlite-utils/blob/9c67cb925253cd5ef54a1fe0496e0ff9caeacfd6/sqlite_utils/cli.py#L1266-L1277
Running it from a SQL query makes it easy to support modifiers like `--csv` and `-t`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906356331,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853541869,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853541869,MDEyOklzc3VlQ29tbWVudDg1MzU0MTg2OQ==,9599,2021-06-03T03:54:14Z,2021-06-03T03:54:14Z,OWNER,Documentation: https://sqlite-utils.datasette.io/en/latest/python-api.html#xindexes,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853535559,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853535559,MDEyOklzc3VlQ29tbWVudDg1MzUzNTU1OQ==,9599,2021-06-03T03:32:47Z,2021-06-03T03:32:47Z,OWNER,"New design:
```python
def test_xindexes(fresh_db):
fresh_db.executescript(
""""""
create table Gosh (c1 text, c2 text, c3 text);
create index Gosh_c1 on Gosh(c1);
create index Gosh_c2c3 on Gosh(c2, c3 desc);
""""""
)
assert fresh_db[""Gosh""].xindexes == [
XIndex(
name=""Gosh_c2c3"",
columns=[
XIndexColumn(seqno=0, cid=1, name=""c2"", desc=0, coll=""BINARY"", key=1),
XIndexColumn(seqno=1, cid=2, name=""c3"", desc=1, coll=""BINARY"", key=1),
XIndexColumn(seqno=2, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
],
),
XIndex(
name=""Gosh_c1"",
columns=[
XIndexColumn(seqno=0, cid=0, name=""c1"", desc=0, coll=""BINARY"", key=1),
XIndexColumn(seqno=1, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
],
),
]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853534732,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853534732,MDEyOklzc3VlQ29tbWVudDg1MzUzNDczMg==,9599,2021-06-03T03:30:10Z,2021-06-03T03:30:10Z,OWNER,"I'm going to return `XIndex(name, columns)` - where `columns` is a list of `XIndexColumn`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853530348,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853530348,MDEyOklzc3VlQ29tbWVudDg1MzUzMDM0OA==,9599,2021-06-03T03:16:33Z,2021-06-03T03:16:33Z,OWNER,"In prototyping this out I realize that I actually want to get back the name of each index, then for each of them the detailed list of index columns. Here's the test from my initial prototype:
```python
def test_xindexes(fresh_db):
fresh_db.executescript(
""""""
create table Gosh (c1 text, c2 text, c3 text);
create index Gosh_c1 on Gosh(c1);
create index Gosh_c2c3 on Gosh(c2, c3 desc);
""""""
)
assert fresh_db[""Gosh""].xindexes == [
(
""Gosh_c2c3"",
[
XIndex(seqno=0, cid=1, name=""c2"", desc=0, coll=""BINARY"", key=1),
XIndex(seqno=1, cid=2, name=""c3"", desc=1, coll=""BINARY"", key=1),
XIndex(seqno=2, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
],
),
(
""Gosh_c1"",
[
XIndex(seqno=0, cid=0, name=""c1"", desc=0, coll=""BINARY"", key=1),
XIndex(seqno=1, cid=-1, name=None, desc=0, coll=""BINARY"", key=0),
],
),
]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,
https://github.com/simonw/sqlite-utils/issues/261#issuecomment-853525036,https://api.github.com/repos/simonw/sqlite-utils/issues/261,853525036,MDEyOklzc3VlQ29tbWVudDg1MzUyNTAzNg==,9599,2021-06-03T03:02:22Z,2021-06-03T03:02:22Z,OWNER,"This would be a breaking change - and the fact that it returns auxiliary columns isn't particularly useful for most cases - ""Auxiliary columns are additional columns needed to locate the table entry that corresponds to each index entry"".
Instead, I'm going to add a new property `table.xindexes` which exposes this.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",906345899,