html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,issue,performed_via_github_app
https://github.com/simonw/sqlite-utils/issues/425#issuecomment-1129332959,https://api.github.com/repos/simonw/sqlite-utils/issues/425,1129332959,IC_kwDOCGYnMM5DUEDf,102771161,2022-05-17T21:27:02Z,2022-05-17T21:27:02Z,NONE,"Hi, I'm trying to deploy my site using elasticbeanstalk and I keep getting this same error :
deterministic=True requires SQLite 3.8.3 or higher

I saw your previous solution that involves editing sqlite-utils/sqlite_utils/db.py file, but I'm curious as to how that will work in production.","{""total_count"": 5, ""+1"": 5, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1203842656,
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556190531,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556190531,IC_kwDOCGYnMM5cwZVD,9599,2023-05-21T14:13:43Z,2023-05-21T14:13:43Z,OWNER,"OK, this works!

![trogon](https://github.com/simonw/sqlite-utils/assets/9599/2ae194c5-ec82-471a-9d1b-b01b3f2632f3)

To try it out, install that branch from GitHub:

    pip install https://github.com/simonw/sqlite-utils/archive/refs/heads/trogon.zip

Then run this:

    sqlite-utils install trogon

And this:

    sqlite-utils tui
","{""total_count"": 5, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 3, ""eyes"": 0}",1718517882,
https://github.com/dogsheep/dogsheep-photos/pull/31#issuecomment-1190995982,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/31,1190995982,IC_kwDOD079W85G_SgO,19231792,2022-07-21T03:26:38Z,2023-04-14T22:41:31Z,NONE,"👋 Any update on getting this merged?

Alternatively, is there a work around for this issue to unblock myself?

edit to add: huge fan of both this project and `osxphotos`, thanks so much for your work here 🙏  If I had any experience with Python I would offer to help but somehow I've managed to not write any Python in 10+ years of programming 😅 

Edit again to add:

> Alternatively, is there a work around for this issue to unblock myself?

Yes, there is. I was able to apply the patch of this PR and it applies (mostly) cleanly and works.

- verified I have a high enough version of `osxphotos`
- downloaded the .patch of this (by appending `.patch` to the URL)
- edited the patch to remove the `setup.py` changes
- `cd` to the directory containing `dogsheep-photos` and `git apply 31.patch`
","{""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",771511344,
https://github.com/simonw/datasette/issues/782#issuecomment-712585687,https://api.github.com/repos/simonw/datasette/issues/782,712585687,MDEyOklzc3VlQ29tbWVudDcxMjU4NTY4Nw==,9599,2020-10-20T04:47:02Z,2020-10-20T04:47:12Z,OWNER,"Great point about CORS, I hadn't considered that.

I think I'm going to keep the `Link:` header (added in #1014) because I quite enjoy using it with GitHub and WordPress, but I'm not going to have it be the default way of doing pagination. For the default shape I'm now leaning towards this:

```json
{
  ""total"": 36,
  ""rows"": [{""id"": 1, ""name"": ""Cleo""}],
  ""next_url"": ""https://latest-with-plugins.datasette.io/fixtures/facetable.json?_next=5""
}
```

So three keys: `total`, `rows` and `next_url`. Then extra keys can be added using `?_extra=` with various named bundles.","{""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",627794879,
https://github.com/dogsheep/google-takeout-to-sqlite/pull/5#issuecomment-791530093,https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5,791530093,MDEyOklzc3VlQ29tbWVudDc5MTUzMDA5Mw==,306240,2021-03-05T16:28:07Z,2021-03-05T16:28:07Z,NONE,"> I just tried to run this on a small VPS instance with 2GB of memory and it crashed out of memory while processing a 12GB mbox from Takeout.
> 
> Is it possible to stream the emails to sqlite instead of loading it all into memory and upserting at once?

@maxhawkins a limitation of the python mbox module is it loads the entire mbox into memory.  I did find another approach to this problem that didn't use the builtin python mbox module and created a generator so that it didn't have to load the whole mbox into memory.  I was hoping to use standard library modules, but this might be a good reason to investigate that approach a bit more. My worry is making sure a custom processor handles all the ins and outs of the mbox format correctly.

Hm. As I'm writing this, I thought of something.  I think I can parse each message one at a time, and then use an mbox function to load each message using the python mbox module. That way the mbox module can still deal with the specifics of the mbox format, but I can use a generator.

I'll give that a try.  Thanks for the feedback @maxhawkins and @simonw. I'll give that a try.

@simonw can we hold off on merging this until I can test this new approach?","{""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",813880401,
https://github.com/simonw/datasette/issues/1258#issuecomment-808651088,https://api.github.com/repos/simonw/datasette/issues/1258,808651088,MDEyOklzc3VlQ29tbWVudDgwODY1MTA4OA==,9599,2021-03-27T04:41:52Z,2021-03-27T04:42:14Z,OWNER,"Right now they look like this:
```yaml
databases:
  fixtures:
    queries:
      neighborhood_search:
        params:
        - text
```
In addition to being able to specify defaults, I'd also like to add other things in the future - most significantly the ability to specify a different input widget (e.g. textarea v.s. single-line input)

So maybe this looks like:
```yaml
params:
- name: text
  default: """"
- name: age
  widget: number
```","{""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",828858421,
https://github.com/simonw/datasette/issues/670#issuecomment-849022714,https://api.github.com/repos/simonw/datasette/issues/670,849022714,MDEyOklzc3VlQ29tbWVudDg0OTAyMjcxNA==,9599,2021-05-26T18:33:47Z,2021-05-26T18:33:58Z,OWNER,Worth mentioning here: I've been doing a tun of research around running Datasette-like functionality against PostgreSQL in my https://github.com/simonw/django-sql-dashboard project - which will definitely inform the Datasette implementation.,"{""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",564833696,
https://github.com/simonw/sqlite-utils/issues/348#issuecomment-1011855133,https://api.github.com/repos/simonw/sqlite-utils/issues/348,1011855133,IC_kwDOCGYnMM48T68d,9599,2022-01-13T07:06:59Z,2022-01-13T07:06:59Z,OWNER,Wrote a lot more about this feature here: https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/,"{""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1067771698,
https://github.com/simonw/sqlite-utils/issues/403#issuecomment-1032987901,https://api.github.com/repos/simonw/sqlite-utils/issues/403,1032987901,IC_kwDOCGYnMM49kiT9,9599,2022-02-08T19:36:06Z,2022-02-08T19:36:06Z,OWNER,New documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#adding-a-primary-key-to-a-rowid-table,"{""total_count"": 3, ""+1"": 3, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1126692066,
https://github.com/simonw/datasette/issues/93#issuecomment-344440377,https://api.github.com/repos/simonw/datasette/issues/93,344440377,MDEyOklzc3VlQ29tbWVudDM0NDQ0MDM3Nw==,9599,2017-11-14T23:56:35Z,2017-11-14T23:56:35Z,OWNER,"It worked!

    $ pyinstaller -F \
        --add-data /usr/local/lib/python3.5/site-packages/datasette/templates:datasette/templates \
        --add-data /usr/local/lib/python3.5/site-packages/datasette/static:datasette/static \
        /usr/local/bin/datasette

    $ file dist/datasette 
    dist/datasette: Mach-O 64-bit executable x86_64
    $ dist/datasette --help
    Usage: datasette [OPTIONS] COMMAND [ARGS]...

      Datasette!

    Options:
      --help  Show this message and exit.

    Commands:
      serve*   Serve up specified SQLite database files with...
      build
      package  Package specified SQLite files into a new...
      publish  Publish specified SQLite database files to...
","{""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 3, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",273944952,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1223378004,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1223378004,IC_kwDOCGYnMM5I60RU,1558033,2022-08-23T00:44:11Z,2022-08-23T00:44:11Z,NONE,"This bug affects me as well. Env:

```
Python 3.8.12
sqlite-utils, version 3.28
sqlite3 3.32.3
MacOS Big Sur 11.6.7
Intel
```

Similar to @mdrovdahl, I was able to work around this bug by piping the SQL string constructed in `add_foreign_keys` to the `sqlite3` command itself. Specifically, if you're trying to patch this yourself, replace [lines 1026-1039 of db.py in your site packages](https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/db.py#L1026-L1039) with something similar to the following:

```
print(""PRAGMA writable_schema = 1;"")
for table_name, new_sql in table_sql.items():
    print(""UPDATE sqlite_master SET sql = '{}' WHERE name = '{}';"".format(
         new_sql, table_name)
    )
print(""PRAGMA writable_schema = 0;"")
print(""VACUUM;"")
```
 
Then from your terminal:
`db-to-sqlite ""<connection string>""  your.db --all > output.sql && sqlite3 your.db < output.sql`

If you want to run this with `-p`, you'll have to actually open a file in code to write to instead of redirecting the output.","{""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 3, ""rocket"": 0, ""eyes"": 0}",810618495,
https://github.com/simonw/datasette/issues/1886#issuecomment-1313252879,https://api.github.com/repos/simonw/datasette/issues/1886,1313252879,IC_kwDOBm6k_c5ORqYP,883348,2022-11-14T08:10:23Z,2022-11-14T08:10:23Z,CONTRIBUTOR,"Hi @simonw and thanks for the great tools you're publishing, your dedication is inspiring!

I work for the French Ministry of Culture on a surveying tool for objects protected for their historical value. It is part of a program building modern public services called [beta.gouv.fr](https://beta.gouv.fr/).

In that context I'm using data published by the Ministry that I have ingested into datasette and published on a free Fly instance : https://collectif-objets-datasette.fly.dev . I have also ingested another data set with infos about french cities on this instance so that I can perform joined queries.

The surveying tool synchronizes its data regularly from this datasette instance, and I also use it to perform queries when asked generic questions about the distribution of objects. (The data is not very accessible as it's undocumented and for internal usage mostly)","{""total_count"": 3, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 3, ""rocket"": 0, ""eyes"": 0}",1447050738,
https://github.com/simonw/datasette/issues/417#issuecomment-473312514,https://api.github.com/repos/simonw/datasette/issues/417,473312514,MDEyOklzc3VlQ29tbWVudDQ3MzMxMjUxNA==,9599,2019-03-15T14:42:07Z,2019-03-17T22:12:30Z,OWNER,"A neat ability of Datasette Library would be if it can work against other files that have been dropped into the folder. In particular: if a user drops a CSV file into the folder, how about automatically converting that CSV file to SQLite using [sqlite-utils](https://github.com/simonw/sqlite-utils)?","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421546944,
https://github.com/simonw/datasette/issues/431#issuecomment-488555399,https://api.github.com/repos/simonw/datasette/issues/431,488555399,MDEyOklzc3VlQ29tbWVudDQ4ODU1NTM5OQ==,9599,2019-05-02T05:13:54Z,2019-05-02T05:13:54Z,OWNER,"Datasette master now treats databases as readonly but NOT immutable. This means you can make changes to those databases from another process and those changes will be instantly reflected in the Datasette interface.

As such, reloading on database change is no longer necessary. Closing this ticket.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",432870248,
https://github.com/simonw/datasette/issues/662#issuecomment-580028669,https://api.github.com/repos/simonw/datasette/issues/662,580028669,MDEyOklzc3VlQ29tbWVudDU4MDAyODY2OQ==,9599,2020-01-30T00:30:19Z,2020-01-30T00:30:19Z,OWNER,I just shipped 0.34: https://datasette.readthedocs.io/en/stable/changelog.html#v0-34,"{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",556814876,
https://github.com/simonw/sqlite-utils/issues/86#issuecomment-586729798,https://api.github.com/repos/simonw/sqlite-utils/issues/86,586729798,MDEyOklzc3VlQ29tbWVudDU4NjcyOTc5OA==,9599,2020-02-16T17:11:02Z,2020-02-16T17:11:02Z,OWNER,I filed a bug in the Python issue tracker here: https://bugs.python.org/issue39652,"{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",564579430,
https://github.com/dogsheep/dogsheep-photos/issues/4#issuecomment-615932007,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4,615932007,MDEyOklzc3VlQ29tbWVudDYxNTkzMjAwNw==,9599,2020-04-18T19:27:55Z,2020-04-18T19:27:55Z,MEMBER,"Research thread: https://twitter.com/simonw/status/1249049694984011776
> I want to build some software that lets people store their own data in their own S3 bucket, but if possible I'd like not to have to teach people the incantations needed to get their bucket setup and minimum-permission credentials figures out

https://testdriven.io/blog/storing-django-static-and-media-files-on-amazon-s3/ looks useful","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",602533539,
https://github.com/simonw/datasette/issues/670#issuecomment-696163452,https://api.github.com/repos/simonw/datasette/issues/670,696163452,MDEyOklzc3VlQ29tbWVudDY5NjE2MzQ1Mg==,652285,2020-09-21T14:46:10Z,2020-09-21T14:46:10Z,NONE,I'm currently using PostgREST to serve OpenAPI APIs off Postgresql databases. I would like to try out datasette once this becomes available on Postgres.,"{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",564833696,
https://github.com/simonw/datasette/issues/749#issuecomment-737563699,https://api.github.com/repos/simonw/datasette/issues/749,737563699,MDEyOklzc3VlQ29tbWVudDczNzU2MzY5OQ==,9599,2020-12-02T23:45:42Z,2020-12-02T23:45:42Z,OWNER,"I asked about this on Twitter - https://twitter.com/steren/status/1334281184965140483

> You simply need to send the `Transfer-Encoding: chunked` header.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610829227,
https://github.com/simonw/datasette/issues/417#issuecomment-751504136,https://api.github.com/repos/simonw/datasette/issues/417,751504136,MDEyOklzc3VlQ29tbWVudDc1MTUwNDEzNg==,212369,2020-12-27T19:02:06Z,2020-12-27T19:02:06Z,NONE,"Very much looking forward to seeing this functionality come together. This is probably out-of-scope for an initial release, but in the future it could be useful to also think of how to run this is a container'ized context. For example, an immutable datasette container that points to an S3 bucket of SQLite DBs or CSVs. Or an immutable datasette container pointing to a NFS volume elsewhere on a Kubernetes cluster.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421546944,
https://github.com/simonw/datasette/issues/1101#issuecomment-755133937,https://api.github.com/repos/simonw/datasette/issues/1101,755133937,MDEyOklzc3VlQ29tbWVudDc1NTEzMzkzNw==,9599,2021-01-06T07:25:48Z,2021-01-06T07:26:43Z,OWNER,"Idea: instead of returning a dictionary, `register_output_renderer` could return an object. The object could have the following properties:

- `.extension` - the extension to use
- `.can_render(...)` - says if it can render this
- `.can_stream(...)` - says if streaming is supported
- `async .stream_rows(rows_iterator, send)` - method that loops through all rows and uses `send` to send them to the response in the correct format

I can then deprecate the existing `dict` return type for 1.0.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",749283032,
https://github.com/simonw/datasette/issues/782#issuecomment-782765665,https://api.github.com/repos/simonw/datasette/issues/782,782765665,MDEyOklzc3VlQ29tbWVudDc4Mjc2NTY2NQ==,9599,2021-02-20T23:34:41Z,2021-02-20T23:34:41Z,OWNER,"OK, I'm back to the ""top level object as the default"" side of things now - it's pretty much unanimous at this point, and it's certainly true that it's not a decision you'll even regret.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",627794879,
https://github.com/simonw/datasette/issues/1262#issuecomment-802099264,https://api.github.com/repos/simonw/datasette/issues/1262,802099264,MDEyOklzc3VlQ29tbWVudDgwMjA5OTI2NA==,9599,2021-03-18T16:43:09Z,2021-03-18T16:43:09Z,OWNER,"I often find myself wanting this too, when I'm exploring a new dataset.

i agree with Bob that this is a good candidate for a plugin. The plugin system isn't quite setup for this yet though - there isn't an obvious mechanism for adding extra sort orders or other interface elements that manipulate the query used by the table view in some way.

I'm going to promote this issue to status of a plugin hook feature request - I have a hunch that a plugin hook that enables `order by random()` could enable a lot of other useful plugin features too.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",834602299,
https://github.com/simonw/datasette/issues/1388#issuecomment-875738149,https://api.github.com/repos/simonw/datasette/issues/1388,875738149,MDEyOklzc3VlQ29tbWVudDg3NTczODE0OQ==,9599,2021-07-07T16:14:29Z,2021-07-07T16:14:29Z,OWNER,This sounds like a valuable feature for people running Datasette behind a proxy.,"{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",939051549,
https://github.com/simonw/datasette/issues/1419#issuecomment-893133496,https://api.github.com/repos/simonw/datasette/issues/1419,893133496,IC_kwDOBm6k_c41PCK4,9599,2021-08-05T03:22:44Z,2021-08-05T03:22:44Z,OWNER,"I ran into this exact same problem today! I only just learned how to use filter on aggregates: https://til.simonwillison.net/sqlite/sqlite-aggregate-filter-clauses

A workaround I used is to add this to the deploy command:

    datasette publish cloudrun ... --install=pysqlite3-binary

This will install the https://pypi.org/project/pysqlite3-binary for package which bundles a more recent SQLite version.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",959710008,
https://github.com/simonw/sqlite-utils/issues/412#issuecomment-1059650190,https://api.github.com/repos/simonw/sqlite-utils/issues/412,1059650190,IC_kwDOCGYnMM4_KPqO,9599,2022-03-05T02:04:43Z,2022-03-05T02:04:54Z,OWNER,"To be honest, I'm having second thoughts about this now mainly because the idiom for turning a generator of dicts into a DataFrame is SO simple:

```python
df = pd.DataFrame(db.query(""select * from articles""))
```
Given it's that simple, I'm questioning if there's any value to adding this to `sqlite-utils` at all. This likely becomes a documentation thing instead!","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160182768,
https://github.com/simonw/sqlite-utils/issues/412#issuecomment-1059652834,https://api.github.com/repos/simonw/sqlite-utils/issues/412,1059652834,IC_kwDOCGYnMM4_KQTi,596279,2022-03-05T02:14:40Z,2022-03-05T02:14:40Z,NONE,"We do a lot of `df.to_sql()` to write into sqlite, mostly in [this moddule](https://github.com/catalyst-cooperative/pudl/blob/main/src/pudl/load.py#L25)","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1160182768,
https://github.com/simonw/sqlite-utils/issues/159#issuecomment-1111506339,https://api.github.com/repos/simonw/sqlite-utils/issues/159,1111506339,IC_kwDOCGYnMM5CQD2j,154364,2022-04-27T21:35:13Z,2022-04-27T21:35:13Z,NONE,"Just stumbled across this, wondering why none of my deletes were working.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",702386948,
https://github.com/simonw/datasette/issues/1727#issuecomment-1112889800,https://api.github.com/repos/simonw/datasette/issues/1727,1112889800,IC_kwDOBm6k_c5CVVnI,9599,2022-04-29T05:29:38Z,2022-04-29T05:29:38Z,OWNER,"OK, I just got the most incredible result with that!

I started up a container running `bash` like this, from my `datasette` checkout. I'm mapping port 8005 on my laptop to port 8001 inside the container because laptop port 8001 was already doing something else:
```
docker run -it --rm --name my-running-script -p 8005:8001 -v ""$PWD"":/usr/src/myapp \
  -w /usr/src/myapp nogil/python bash
```
Then in `bash` I ran the following commands to install Datasette and its dependencies:
```
pip install -e '.[test]'
pip install datasette-pretty-traces # For debug tracing
```
Then I started Datasette against my `github.db` database (from github-to-sqlite.dogsheep.net/github.db) like this:

```
datasette github.db -h 0.0.0.0 --setting trace_debug 1
```
I hit the following two URLs to compare the parallel v.s. not parallel implementations:

- `http://127.0.0.1:8005/github/issues?_facet=milestone&_facet=repo&_trace=1&_size=10`
- `http://127.0.0.1:8005/github/issues?_facet=milestone&_facet=repo&_trace=1&_size=10&_noparallel=1`

And... the parallel one beat the non-parallel one decisively, on multiple page refreshes!

Not parallel: 77ms

Parallel: 47ms

<img width=""1213"" alt=""CleanShot 2022-04-28 at 22 10 54@2x"" src=""https://user-images.githubusercontent.com/9599/165889437-60d4200d-698a-4175-af23-7c03bb456e66.png"">

<img width=""1213"" alt=""CleanShot 2022-04-28 at 22 10 21@2x"" src=""https://user-images.githubusercontent.com/9599/165889445-2dfb8676-d823-405e-aecb-ad28ec3043da.png"">

So yeah, I'm very confident this is a problem with the GIL. And I am absolutely **stunned** that @colesbury's fork ran Datasette (which has some reasonably tricky threading and async stuff going on) out of the box!","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1217759117,
https://github.com/simonw/sqlite-utils/issues/456#issuecomment-1190449764,https://api.github.com/repos/simonw/sqlite-utils/issues/456,1190449764,IC_kwDOCGYnMM5G9NJk,45919695,2022-07-20T15:45:54Z,2022-07-20T15:45:54Z,NONE,"> hadley wickham's melt and reshape could be good inspo: http://had.co.nz/reshape/introduction.pdf

Note that Hadley has since implemented `pivot_longer` and `pivot_wider` instead of the previous verbs/functions that he used. Those can be found in the tidyr package and are probably the best reference which includes all of the learnings from years of user feedback. https://tidyr.tidyverse.org/articles/pivot.html","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1310243385,
https://github.com/simonw/sqlite-utils/issues/239#issuecomment-1236214402,https://api.github.com/repos/simonw/sqlite-utils/issues/239,1236214402,IC_kwDOCGYnMM5JryKC,9599,2022-09-03T23:46:02Z,2022-09-03T23:46:02Z,OWNER,Yeah having a version of this that can setup m2m relationships would definitely be interesting.,"{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",816526538,
https://github.com/simonw/sqlite-utils/issues/448#issuecomment-1297703307,https://api.github.com/repos/simonw/sqlite-utils/issues/448,1297703307,IC_kwDOCGYnMM5NWWGL,167893,2022-10-31T21:23:51Z,2022-10-31T21:27:32Z,CONTRIBUTOR,"The Windows aspect is a red herring: OP's sample above produces the same error on Linux. (Though I don't know what's going on with the CI).

The same error can also be obtained by passing an `io` from a file opened in non-binary mode (`'r'` as opposed to `'rb'`) to `rows_from_file()`. This is how I got here.

The fix for my case is easy: open the file in mode `'rb'`. The analagous fix for OP's problem also works: use `BytesIO` in place of `StringIO`.

Minimal test case (derived from [utils.py](https://github.com/simonw/sqlite-utils/blob/main/sqlite_utils/utils.py#L304)):

``` python
import io
from typing import cast

#fp = io.StringIO(""id,name\n1,Cleo"") # error
fp = io.BytesIO(bytes(""id,name\n1,Cleo"", encoding='utf-8')) # okay
reader = io.BufferedReader(cast(io.RawIOBase, fp))
reader.peek(1) # exception thrown here
```
I see the signature of `rows_from_file()` correctly has `fp: BinaryIO` but I guess you'd need either a runtime type check for that (not all `io`s have `mode()`), or to catch the `AttributeError` on `peek()` to produce a better error for users. Neither option is ideal.

Some thoughts on testing binary-ness of `io`s in this SO question: https://stackoverflow.com/questions/44584829/how-to-determine-if-file-is-opened-in-binary-or-text-mode","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1279144769,
https://github.com/simonw/datasette/issues/661#issuecomment-579830682,https://api.github.com/repos/simonw/datasette/issues/661,579830682,MDEyOklzc3VlQ29tbWVudDU3OTgzMDY4Mg==,9599,2020-01-29T16:07:41Z,2020-01-29T16:07:41Z,OWNER,Having `datasette package` take an optional port argument seems like a good feature to me.,"{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",555832585,
https://github.com/simonw/sqlite-utils/issues/139#issuecomment-683178570,https://api.github.com/repos/simonw/sqlite-utils/issues/139,683178570,MDEyOklzc3VlQ29tbWVudDY4MzE3ODU3MA==,9599,2020-08-28T22:48:51Z,2020-08-28T22:48:51Z,OWNER,"Thanks @simonwiles, this is now released in 2.16.1: https://sqlite-utils.readthedocs.io/en/stable/changelog.html","{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",686978131,
https://github.com/simonw/datasette/issues/1143#issuecomment-746827083,https://api.github.com/repos/simonw/datasette/issues/1143,746827083,MDEyOklzc3VlQ29tbWVudDc0NjgyNzA4Mw==,9599,2020-12-16T18:56:07Z,2020-12-16T18:56:07Z,OWNER,"I think the right way to do this is to support multiple optional `--cors-origin=` pattern values, like you suggested.","{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 1, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",764059235,
https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229125114,https://api.github.com/repos/simonw/sqlite-utils/issues/471,1229125114,IC_kwDOCGYnMM5JQvX6,9599,2022-08-27T05:08:58Z,2022-08-27T05:08:58Z,OWNER,"Testing `bulk --functions`:

```
% sqlite-utils create-table chickens.db chickens id integer name text name_upper text 
% echo 'id,name
1,Blue
2,Snowy
3,Azi
4,Lila
5,Suna
6,Cardi' | sqlite-utils bulk chickens.db '
insert into chickens (id, name, name_upper) values (:id, :name, myupper(:name))
' - --functions '
def myupper(s):
    return s.upper()
' --csv
% sqlite-utils rows chickens.db chickens
[{""id"": 1, ""name"": ""Blue"", ""name_upper"": ""BLUE""},
 {""id"": 2, ""name"": ""Snowy"", ""name_upper"": ""SNOWY""},
 {""id"": 3, ""name"": ""Azi"", ""name_upper"": ""AZI""},
 {""id"": 4, ""name"": ""Lila"", ""name_upper"": ""LILA""},
 {""id"": 5, ""name"": ""Suna"", ""name_upper"": ""SUNA""},
 {""id"": 6, ""name"": ""Cardi"", ""name_upper"": ""CARDI""}]
```","{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1352932716,
https://github.com/simonw/sqlite-utils/issues/545#issuecomment-1556191894,https://api.github.com/repos/simonw/sqlite-utils/issues/545,1556191894,IC_kwDOCGYnMM5cwZqW,9599,2023-05-21T14:20:14Z,2023-05-21T14:20:14Z,OWNER,"Opened a feature request for customizing the help and command name:

- https://github.com/Textualize/trogon/issues/2","{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",1718517882,
https://github.com/simonw/datasette/issues/969#issuecomment-705874457,https://api.github.com/repos/simonw/datasette/issues/969,705874457,MDEyOklzc3VlQ29tbWVudDcwNTg3NDQ1Nw==,9599,2020-10-08T23:27:30Z,2020-10-08T23:27:30Z,OWNER,For the moment I'm going to ship this as the `--tar=` option. Can consider detecting `gtar` in the future.,"{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 1, ""eyes"": 0}",705057955,
https://github.com/simonw/datasette/pull/2052#issuecomment-1530822437,https://api.github.com/repos/simonw/datasette/issues/2052,1530822437,IC_kwDOBm6k_c5bPn8l,193185,2023-05-02T03:35:30Z,2023-05-02T16:02:38Z,CONTRIBUTOR,"Also, just checking - is this how I'd write bulletproof plugin registration code that is robust against the order in which the script tags load (eg if both my code and the Datasette code are loaded via a `<script async src='...'/>` tag)?

```js
if (window.__DATASETTE__)
  go(window.__DATASETTE__);
else
  document.addEventListener(""datasette_init"", (evt) => go(evt.detail));

function go(manager) {
  manager.registerPlugin(...)
}
```

I don't know if it'd make sense, but you could also consider the asynchronous queuing pattern that Google Analytics uses (see [this Stack Overflow post](https://stackoverflow.com/questions/6963779/whats-the-name-of-google-analytics-async-design-pattern-and-where-is-it-used) for more details):

```js
__DATASETTE__ = __DATASETTE__ || [];
__DATASETTE__.push(go);

function go(manager) {
  manager.registerPlugin(...);
}
```","{""total_count"": 2, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 1}",1651082214,
https://github.com/simonw/datasette/pull/1159#issuecomment-1658968449,https://api.github.com/repos/simonw/datasette/issues/1159,1658968449,IC_kwDOBm6k_c5i4dmB,9599,2023-07-31T18:57:58Z,2023-07-31T18:57:58Z,OWNER,"Added to the 1.0 milestone because: https://news.ycombinator.com/item?id=36932876#36939734

>  Hah, wow, it looks like I've been procrastinating on making a decision if I like that or not for three years!
>
> I'll add it to the Datasette 1.0 milestone so it definitely gets my attention before shipping that release. ","{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 1, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 1, ""eyes"": 0}",774332247,
https://github.com/dogsheep/healthkit-to-sqlite/issues/7#issuecomment-514500253,https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/7,514500253,MDEyOklzc3VlQ29tbWVudDUxNDUwMDI1Mw==,9599,2019-07-24T06:34:28Z,2019-07-24T06:34:28Z,MEMBER,"Clearing the root element each time saved even more:

<img width=""1128"" alt=""Screen Shot 2019-07-24 at 8 30 38 AM"" src=""https://user-images.githubusercontent.com/9599/61770555-d3932100-aded-11e9-8ffe-bebd682f94ed.png"">
","{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",472097220,
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622162835,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622162835,MDEyOklzc3VlQ29tbWVudDYyMjE2MjgzNQ==,9599,2020-04-30T22:59:26Z,2020-04-30T22:59:26Z,MEMBER,Documentation: https://github.com/dogsheep/github-to-sqlite/blob/c9f48404481882e8b3af06f35e4801a80ac79ed6/README.md#scraping-dependents-for-a-repository,"{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,
https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1606411508,https://api.github.com/repos/simonw/sqlite-utils/issues/235,1606411508,IC_kwDOCGYnMM5fv-T0,9599,2023-06-26T01:42:10Z,2023-06-26T01:42:22Z,OWNER,https://sqlite-utils.datasette.io/en/stable/changelog.html#v3-33 - upgrading to `sqlite-utils>=3.33` and then installing both `sqlean.py` and `sqlite-dump` in the same virtual environment as `sqlite-utils` should fix this issue.,"{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 2, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",810618495,
https://github.com/simonw/datasette/issues/176#issuecomment-356175667,https://api.github.com/repos/simonw/datasette/issues/176,356175667,MDEyOklzc3VlQ29tbWVudDM1NjE3NTY2Nw==,4313116,2018-01-09T04:19:03Z,2018-01-09T04:19:03Z,NONE,"@yozlet Yes I think that I was confused when I posted my original comment. I see your main point now and am in agreement.

","{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 2, ""rocket"": 0, ""eyes"": 0}",285168503,
https://github.com/simonw/datasette/issues/176#issuecomment-548508237,https://api.github.com/repos/simonw/datasette/issues/176,548508237,MDEyOklzc3VlQ29tbWVudDU0ODUwODIzNw==,634572,2019-10-31T18:25:44Z,2019-10-31T18:25:44Z,NONE,"👋 I'd be interested in building this out in Q1 or Q2 of 2020 if nobody has tackled it by then. I would love to integrate Datasette into @thechicagoreporter's practice, but we're also fully committed to GraphQL moving forward.","{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 2, ""rocket"": 0, ""eyes"": 0}",285168503,
https://github.com/simonw/datasette/issues/1134#issuecomment-742260116,https://api.github.com/repos/simonw/datasette/issues/1134,742260116,MDEyOklzc3VlQ29tbWVudDc0MjI2MDExNg==,2181410,2020-12-10T05:57:17Z,2020-12-10T05:57:17Z,NONE,"Hi Simon

Thank you for the quick fix! And glad you like our use of Datasette (launches 1. january 2021). It's a site that currently (more to come) makes all minutes and their annexes from Aarhus City Council and the major committees (1997-2019) available to the public. So we're putting Datasette to good use :)","{""total_count"": 2, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 2, ""rocket"": 0, ""eyes"": 0}",760312579,
https://github.com/simonw/datasette/issues/1175#issuecomment-762488336,https://api.github.com/repos/simonw/datasette/issues/1175,762488336,MDEyOklzc3VlQ29tbWVudDc2MjQ4ODMzNg==,758858,2021-01-18T21:59:28Z,2021-01-18T22:00:31Z,NONE,"I encountered your issue when trying to find a solution and came up with the following, maybe it can help.

```python
import logging.config
from typing import Tuple

import structlog
import uvicorn

from example.config import settings

shared_processors: Tuple[structlog.types.Processor, ...] = (
    structlog.contextvars.merge_contextvars,
    structlog.stdlib.add_logger_name,
    structlog.stdlib.add_log_level,
    structlog.processors.TimeStamper(fmt=""iso""),
)

logging_config = {
    ""version"": 1,
    ""disable_existing_loggers"": False,
    ""formatters"": {
        ""json"": {
            ""()"": structlog.stdlib.ProcessorFormatter,
            ""processor"": structlog.processors.JSONRenderer(),
            ""foreign_pre_chain"": shared_processors,
        },
        ""console"": {
            ""()"": structlog.stdlib.ProcessorFormatter,
            ""processor"": structlog.dev.ConsoleRenderer(),
            ""foreign_pre_chain"": shared_processors,
        },
        **uvicorn.config.LOGGING_CONFIG[""formatters""],
    },
    ""handlers"": {
        ""default"": {
            ""level"": ""DEBUG"",
            ""class"": ""logging.StreamHandler"",
            ""formatter"": ""json"" if not settings.debug else ""console"",
        },
        ""uvicorn.access"": {
            ""level"": ""INFO"",
            ""class"": ""logging.StreamHandler"",
            ""formatter"": ""access"",
        },
        ""uvicorn.default"": {
            ""level"": ""INFO"",
            ""class"": ""logging.StreamHandler"",
            ""formatter"": ""default"",
        },
    },
    ""loggers"": {
        """": {""handlers"": [""default""], ""level"": ""INFO""},
        ""uvicorn.error"": {
            ""handlers"": [""default"" if not settings.debug else ""uvicorn.default""],
            ""level"": ""INFO"",
            ""propagate"": False,
        },
        ""uvicorn.access"": {
            ""handlers"": [""default"" if not settings.debug else ""uvicorn.access""],
            ""level"": ""INFO"",
            ""propagate"": False,
        },
    },
}


def setup_logging() -> None:
    structlog.configure(
        processors=[
            structlog.stdlib.filter_by_level,
            *shared_processors,
            structlog.stdlib.PositionalArgumentsFormatter(),
            structlog.processors.StackInfoRenderer(),
            structlog.processors.format_exc_info,
            structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
        ],
        context_class=dict,
        logger_factory=structlog.stdlib.LoggerFactory(),
        wrapper_class=structlog.stdlib.AsyncBoundLogger,
        cache_logger_on_first_use=True,
    )
    logging.config.dictConfig(logging_config)
```

And then it'll be run on the startup event:
```python
@app.on_event(""startup"")
async def startup_event() -> None:
    setup_logging()
```

It depends on a setting called `debug` which controls if it should output the regular uvicorn logging or json. ","{""total_count"": 15, ""+1"": 7, ""-1"": 0, ""laugh"": 1, ""hooray"": 1, ""confused"": 0, ""heart"": 5, ""rocket"": 1, ""eyes"": 0}",779156520,
https://github.com/simonw/datasette/issues/46#issuecomment-344161226,https://api.github.com/repos/simonw/datasette/issues/46,344161226,MDEyOklzc3VlQ29tbWVudDM0NDE2MTIyNg==,9599,2017-11-14T06:41:21Z,2017-11-14T06:41:21Z,OWNER,Spatial extensions would be really useful too. https://www.gaia-gis.it/spatialite-2.1/SpatiaLite-manual.html,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",271301468,
https://github.com/simonw/datasette/issues/175#issuecomment-353424169,https://api.github.com/repos/simonw/datasette/issues/175,353424169,MDEyOklzc3VlQ29tbWVudDM1MzQyNDE2OQ==,9599,2017-12-21T18:33:55Z,2017-12-21T18:33:55Z,OWNER,Done - thanks for curating these: https://github.com/topics/automatic-api,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",282971961,
https://github.com/simonw/datasette/issues/97#issuecomment-392602334,https://api.github.com/repos/simonw/datasette/issues/97,392602334,MDEyOklzc3VlQ29tbWVudDM5MjYwMjMzNA==,9599,2018-05-28T20:57:21Z,2018-05-28T20:57:21Z,OWNER,"The `/.json` endpoint is more of an implementation detail of the homepage at this point. A better, documented ( http://datasette.readthedocs.io/en/stable/introspection.html#inspect ) endpoint for finding all of the databases and tables is https://parlgov.datasettes.com/-/inspect.json","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",274022950,
https://github.com/simonw/datasette/issues/339#issuecomment-404565566,https://api.github.com/repos/simonw/datasette/issues/339,404565566,MDEyOklzc3VlQ29tbWVudDQwNDU2NTU2Ng==,9599,2018-07-12T16:08:42Z,2018-07-12T16:08:42Z,OWNER,I'm going to turn this into an issue about better supporting the above option.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",340396247,
https://github.com/simonw/datasette/issues/308#issuecomment-405971920,https://api.github.com/repos/simonw/datasette/issues/308,405971920,MDEyOklzc3VlQ29tbWVudDQwNTk3MTkyMA==,9599,2018-07-18T15:27:12Z,2018-07-18T15:27:12Z,OWNER,"It looks like there are a few extra options we should support:

https://devcenter.heroku.com/articles/heroku-cli-commands

```
  -t, --team=team            team to use
  --region=region            specify region for the app to run in
  --space=space              the private space to create the app in
```

Since these differ from the options for Zeit Now I think this means splitting up `datasette publish now` and `datasette publish Heroku` into separate subcommands.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",330826972,
https://github.com/simonw/datasette/issues/176#issuecomment-431867885,https://api.github.com/repos/simonw/datasette/issues/176,431867885,MDEyOklzc3VlQ29tbWVudDQzMTg2Nzg4NQ==,634572,2018-10-22T15:24:57Z,2018-10-22T15:24:57Z,NONE,"I'd like this as well. It would let me access Datasette-driven projects from GatsbyJS the same way I can access Postgres DBs via Hasura. While I don't see SQLite replacing Postgres for the 50m row datasets I sometimes have to work with, there's a whole class of smaller datasets that are great with Datasette but currently would find another option.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",285168503,
https://github.com/simonw/datasette/issues/370#issuecomment-435974786,https://api.github.com/repos/simonw/datasette/issues/370,435974786,MDEyOklzc3VlQ29tbWVudDQzNTk3NDc4Ng==,9599,2018-11-05T18:06:56Z,2018-11-05T18:06:56Z,OWNER,"I've been thinking a bit about ways of using Jupyter Notebook more effectively with Datasette (thinks like a `publish_dataframes(df1, df2, df3)` function which publishes some Pandas dataframes and returns you a URL to a new hosted Datasette instance) but you're right, Jupyter Lab is potentially a much more interesting fit.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",377155320,
https://github.com/simonw/datasette/issues/120#issuecomment-439421164,https://api.github.com/repos/simonw/datasette/issues/120,439421164,MDEyOklzc3VlQ29tbWVudDQzOTQyMTE2NA==,36796532,2018-11-16T15:05:18Z,2018-11-16T15:05:18Z,NONE,This would be an awesome feature ❤️ ,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",275087397,
https://github.com/simonw/datasette/issues/391#issuecomment-450964512,https://api.github.com/repos/simonw/datasette/issues/391,450964512,MDEyOklzc3VlQ29tbWVudDQ1MDk2NDUxMg==,9599,2019-01-02T19:45:12Z,2019-01-02T19:45:12Z,OWNER,"Thanks, I've fixed this. I had to re-alias it against now:
```
~ $ now alias google-trends-pnwhfwvgqf.now.sh https://google-trends.datasettes.com/
> Assigning alias google-trends.datasettes.com to deployment google-trends-pnwhfwvgqf.now.sh
> Certificate for google-trends.datasettes.com (cert_uXaADIuNooHS3tZ) created [18s]
> Success! google-trends.datasettes.com now points to google-trends-pnwhfwvgqf.now.sh [20s]
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",392610803,
https://github.com/simonw/datasette/issues/397#issuecomment-453330680,https://api.github.com/repos/simonw/datasette/issues/397,453330680,MDEyOklzc3VlQ29tbWVudDQ1MzMzMDY4MA==,9599,2019-01-11T01:17:11Z,2019-01-11T01:25:33Z,OWNER,"If you pull [the latest image](https://hub.docker.com/r/datasetteproject/datasette) you should get the right SQLite version now:

    docker pull datasetteproject/datasette
    docker run -p 8001:8001 \
        datasetteproject/datasette \
        datasette -p 8001 -h 0.0.0.0

http://0.0.0.0:8001/-/versions now gives me:

```
        ""version"": ""3.26.0""
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",397129564,
https://github.com/simonw/datasette/issues/187#issuecomment-467264937,https://api.github.com/repos/simonw/datasette/issues/187,467264937,MDEyOklzc3VlQ29tbWVudDQ2NzI2NDkzNw==,9599,2019-02-26T02:14:28Z,2019-02-26T02:14:28Z,OWNER,I'm working on a port of Datasette to Starlette which I think would fix this issue: https://github.com/encode/starlette,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",309033998,
https://github.com/simonw/datasette/issues/419#issuecomment-473708941,https://api.github.com/repos/simonw/datasette/issues/419,473708941,MDEyOklzc3VlQ29tbWVudDQ3MzcwODk0MQ==,9599,2019-03-17T19:58:11Z,2019-03-17T19:58:11Z,OWNER,"Some problems to solve:

* Right now Datasette assumes it can always show the count of rows in a table, because this has been pre-calculated. If a database is mutable the pre-calculation trick no longer works, and for giant tables a `select count(*) from X` query can be expensive to run. Maybe we set a time limit on these? If time limit expires show ""many rows""?
* Maintaining a content hash of the table no longer makes sense if it is changing (though interestingly there's a `.sha3sum` built-in SQLite CLI command which takes a hash of the content and stays the same even through vacuum runs). Without that we need a different mechanism for calculating table colours. It also means that we can't do the special dbname-hash URL trick (see #418) at all if the database is opened as mutable.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",421551434,
https://github.com/simonw/datasette/issues/486#issuecomment-495659567,https://api.github.com/repos/simonw/datasette/issues/486,495659567,MDEyOklzc3VlQ29tbWVudDQ5NTY1OTU2Nw==,9599,2019-05-24T14:41:45Z,2019-05-24T14:41:45Z,OWNER,"I'm really keen to offer this as a plugin hook once I have Datasette working on ASGI - #272 

I'll hopefully have that working in the next few weeks, but in the meantime there are a couple of tricks you can use:

- you can add static HTML files (no templates though) using the static route configuration options
- you can link to external hosted pages using the `about_url` metadata option
- you can add information to an existing page with a custom template. I do that here for example: https://russian-ira-facebook-ads.datasettes.com/","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",448189298,
https://github.com/simonw/sqlite-utils/issues/21#issuecomment-496786354,https://api.github.com/repos/simonw/sqlite-utils/issues/21,496786354,MDEyOklzc3VlQ29tbWVudDQ5Njc4NjM1NA==,9599,2019-05-29T05:09:01Z,2019-05-29T05:09:01Z,OWNER,Shipped this feature in sqlite-utils 1.1: https://sqlite-utils.readthedocs.io/en/latest/changelog.html#v1-1,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",448391492,
https://github.com/simonw/datasette/issues/498#issuecomment-498839428,https://api.github.com/repos/simonw/datasette/issues/498,498839428,MDEyOklzc3VlQ29tbWVudDQ5ODgzOTQyOA==,9599,2019-06-04T20:53:21Z,2019-06-04T20:53:21Z,OWNER,"It does not, but that's a really great idea for a feature.

One challenge here is that FTS ranking calculations take overall table statistics into account, which means it's usually not possible to combine rankings from different tables in a sensible way. But that doesn't mean it's not possible to return grouped results.

I think this makes a lot of sense as a plugin.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",451513541,
https://github.com/simonw/datasette/issues/499#issuecomment-498840129,https://api.github.com/repos/simonw/datasette/issues/499,498840129,MDEyOklzc3VlQ29tbWVudDQ5ODg0MDEyOQ==,9599,2019-06-04T20:55:30Z,2019-06-04T21:01:22Z,OWNER,"I really want this too!

It's one of the goals of the Datasette Library #417 concept, which I'm hoping to turn into an actual feature in the coming months.

It's also going to be a major focus of my ten month JSK fellowship at Stanford, which starts in September. https://twitter.com/simonw/status/1123624552867565569","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",451585764,
https://github.com/simonw/datasette/issues/527#issuecomment-505057520,https://api.github.com/repos/simonw/datasette/issues/527,505057520,MDEyOklzc3VlQ29tbWVudDUwNTA1NzUyMA==,9599,2019-06-24T15:21:18Z,2019-06-24T15:21:18Z,OWNER,I just released csvs-to-sqlite 0.9.1 with this bug fix: https://github.com/simonw/csvs-to-sqlite/releases/tag/0.9.1,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459936585,
https://github.com/simonw/datasette/pull/437#issuecomment-505087020,https://api.github.com/repos/simonw/datasette/issues/437,505087020,MDEyOklzc3VlQ29tbWVudDUwNTA4NzAyMA==,9599,2019-06-24T16:38:56Z,2019-06-24T16:38:56Z,OWNER,Closing this because it doesn't really fit the new model of inspect (though we should discuss in #465 how to further evolve this feature) and because as-of #272 we no longer use Sanic - though #520 will implement the equivalent of `prepare_sanic` against ASGI.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",438048318,
https://github.com/simonw/datasette/issues/514#issuecomment-509154312,https://api.github.com/repos/simonw/datasette/issues/514,509154312,MDEyOklzc3VlQ29tbWVudDUwOTE1NDMxMg==,4363711,2019-07-08T09:36:25Z,2019-07-08T09:40:33Z,NONE,"@chrismp: Ports 1024 and under are privileged and can usually only be bound by a root or supervisor user, so it makes sense if you're running as the user `chris` that port 8000 works but 80 doesn't.

See [this generic question-and-answer](https://superuser.com/questions/710253/allow-non-root-process-to-bind-to-port-80-and-443) and [this systemd question-and-answer](https://stackoverflow.com/questions/40865775/linux-systemd-service-on-port-80) for more information about ways to skin this cat. Without knowing your specific circumstances, either extending those privileges to that service/executable/user, proxying them through something like nginx or indeed looking at what the nginx systemd job has to do to listen at port 80 all sound like good ways to start.

At this point, this is more generic systemd/Linux support than a Datasette issue, which is why a complete rando like me is able to contribute anything. ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",459397625,
https://github.com/simonw/datasette/pull/556#issuecomment-510550279,https://api.github.com/repos/simonw/datasette/issues/556,510550279,MDEyOklzc3VlQ29tbWVudDUxMDU1MDI3OQ==,9599,2019-07-11T16:07:27Z,2019-07-11T16:07:27Z,OWNER,"This is a really neat trick, thanks!","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",465773546,
https://github.com/simonw/datasette/pull/557#issuecomment-511625212,https://api.github.com/repos/simonw/datasette/issues/557,511625212,MDEyOklzc3VlQ29tbWVudDUxMTYyNTIxMg==,9599,2019-07-16T01:12:14Z,2019-07-16T01:12:14Z,OWNER,This looks useful for dealing with the `The process cannot access the file because it is being used by another process` error: https://stackoverflow.com/a/28032829,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",466996584,
https://github.com/simonw/datasette/pull/595#issuecomment-541931047,https://api.github.com/repos/simonw/datasette/issues/595,541931047,MDEyOklzc3VlQ29tbWVudDU0MTkzMTA0Nw==,9599,2019-10-14T21:25:38Z,2019-10-14T21:25:38Z,OWNER,I like the conditional dependency for the moment - maybe until 3.5 becomes officially unsupported.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506300941,
https://github.com/dogsheep/twitter-to-sqlite/issues/20#issuecomment-544335363,https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/20,544335363,MDEyOklzc3VlQ29tbWVudDU0NDMzNTM2Mw==,9599,2019-10-21T03:32:04Z,2019-10-21T03:32:04Z,MEMBER,"In case anyone is interested, here's an extract from the crontab I'm running these under at the moment:
```
1,11,21,31,41,51 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite user-timeline /home/ubuntu/twitter.db -a /home/ubuntu/auth.json --since
2,7,12,17,22,27,32,37,42,47,52,57 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite home-timeline /home/ubuntu/timeline.db -a /home/ubuntu/auth.json --since
6,16,26,36,46,56 * * * * /home/ubuntu/datasette-venv/bin/twitter-to-sqlite favorites /home/ubuntu/twitter.db -a /home/ubuntu/auth.json --stop_after=50
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506268945,
https://github.com/simonw/sqlite-utils/issues/62#issuecomment-549435364,https://api.github.com/repos/simonw/sqlite-utils/issues/62,549435364,MDEyOklzc3VlQ29tbWVudDU0OTQzNTM2NA==,9599,2019-11-04T16:30:34Z,2019-11-04T16:30:34Z,OWNER,Released as 1.12.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",500783373,
https://github.com/simonw/datasette/issues/567#issuecomment-549665423,https://api.github.com/repos/simonw/datasette/issues/567,549665423,MDEyOklzc3VlQ29tbWVudDU0OTY2NTQyMw==,9599,2019-11-05T05:11:14Z,2019-11-05T05:11:14Z,OWNER,"@clausjuhl I wrote a bit about that here: https://simonwillison.net/2019/May/19/datasette-0-28/

Short version: just point Datasette at a SQLite file and update it from another process - it should work fine! I do it all the time now - I'll have a script running that writes to a database and I'll use Datasette to monitor progress. ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",476573875,
https://github.com/simonw/datasette/pull/595#issuecomment-552275668,https://api.github.com/repos/simonw/datasette/issues/595,552275668,MDEyOklzc3VlQ29tbWVudDU1MjI3NTY2OA==,9599,2019-11-11T03:09:43Z,2019-11-11T03:09:43Z,OWNER,Glitch has been upgraded to Python 3.7. I think I'm happy to drop 3.5 support now - users who want Python 3.5 can get it by installing `datasette==0.30.2`,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",506300941,
https://github.com/simonw/datasette/issues/646#issuecomment-561022224,https://api.github.com/repos/simonw/datasette/issues/646,561022224,MDEyOklzc3VlQ29tbWVudDU2MTAyMjIyNA==,9599,2019-12-03T06:30:42Z,2019-12-03T06:30:42Z,OWNER,"I don't think this is possible at the moment but you're right, it totally should be.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",531502365,
https://github.com/simonw/sqlite-utils/issues/73#issuecomment-570930239,https://api.github.com/repos/simonw/sqlite-utils/issues/73,570930239,MDEyOklzc3VlQ29tbWVudDU3MDkzMDIzOQ==,9599,2020-01-05T17:15:18Z,2020-01-05T17:15:18Z,OWNER,I think this is because you forgot to include a `pk=` argument. I'll change the code to throw a more useful error in this case.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",545407916,
https://github.com/simonw/datasette/issues/662#issuecomment-579787057,https://api.github.com/repos/simonw/datasette/issues/662,579787057,MDEyOklzc3VlQ29tbWVudDU3OTc4NzA1Nw==,9599,2020-01-29T14:43:46Z,2020-01-29T14:43:46Z,OWNER,Can you share the exact queries you're having trouble with? The SQL itself or even just the full URL to the page (it doesn't matter if it's to a Datasette instance that isn't available online - I just need to see the URL parameters).,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",556814876,
https://github.com/simonw/datasette/issues/662#issuecomment-579798917,https://api.github.com/repos/simonw/datasette/issues/662,579798917,MDEyOklzc3VlQ29tbWVudDU3OTc5ODkxNw==,2181410,2020-01-29T15:08:57Z,2020-01-29T15:08:57Z,NONE,"Hi Simon

Thankt you for a quick reply. Here are a few examples of urls, where I search the 'cases_fts'-virtual table for tokens in the title-column. It returns the same results, wether the other query-params are present or not.

Searching for sky
http://localhost:8001/db-7596a4e/cases?_search_title=sky&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date
Returns searchresults

Searching for sky*
http://localhost:8001/db-7596a4e/cases?_search_title=sky*&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date
Returns searchresults

Searching for sky-tog
http://localhost:8001/db-7596a4e/cases?_search_title=sky-tog&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date
Throws: No such column: tog

searching for sky+
http://localhost:8001/db-7596a4e/cases?_search_title=sky%2B&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date
Throws: Invalid SQL: fts5: syntax error near """"

Searching for ""madpakke"" (including double quotes)
http://localhost:8001/db-7596a4e/cases?_search_title=%22madpakke%22&year__gte=1997&year__lte=2017&_sort_desc=last_deliberation_date
Returns searchresults even though 'madpakke' only appears in the fulltextindex without quotes

As I said, my other plugins work just fine, and I just copied your sql_functions.py from the datasette-repo.

Thanks!","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",556814876,
https://github.com/simonw/datasette/issues/662#issuecomment-579832857,https://api.github.com/repos/simonw/datasette/issues/662,579832857,MDEyOklzc3VlQ29tbWVudDU3OTgzMjg1Nw==,9599,2020-01-29T16:12:08Z,2020-01-29T16:12:08Z,OWNER,"I think I see what's happening here.

Adding the new plugin isn't quite enough: the change I made to master also alters the table view code to call the new function:

https://github.com/simonw/datasette/commit/3c861f363df02a59a67c59036278338e4760d2ed#diff-5e0ffd62fced7d46339b9b2cd167c2f9

If you add the escape function as a plugin in Datasette 0.33 you will have to use a custom SQL query to run it, like this:

https://latest.datasette.io/fixtures?sql=select+pk%2C+text1%2C+text2%2C+%5Bname+with+.+and+spaces%5D+from+searchable+where+rowid+in+%28select+rowid+from+searchable_fts+where+searchable_fts+match+escape_fts%28%3Asearch%29%29+order+by+pk+limit+101&search=Dog

Or you can hold out for Datasette 0.34 which will have this fix and will hopefully ship within the next 24 hours.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",556814876,
https://github.com/simonw/datasette/issues/662#issuecomment-579864036,https://api.github.com/repos/simonw/datasette/issues/662,579864036,MDEyOklzc3VlQ29tbWVudDU3OTg2NDAzNg==,2181410,2020-01-29T17:17:01Z,2020-01-29T17:17:01Z,NONE,This is excellent news. I'll wait until version 0.34. It would be tiresome to rewrite all standard-queries into custom queries. Thank you!,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",556814876,
https://github.com/simonw/datasette/issues/661#issuecomment-580028593,https://api.github.com/repos/simonw/datasette/issues/661,580028593,MDEyOklzc3VlQ29tbWVudDU4MDAyODU5Mw==,9599,2020-01-30T00:30:04Z,2020-01-30T00:30:04Z,OWNER,This has now shipped as part of Datasette 0.34: https://datasette.readthedocs.io/en/stable/changelog.html#v0-34,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",555832585,
https://github.com/simonw/datasette/issues/658#issuecomment-580029288,https://api.github.com/repos/simonw/datasette/issues/658,580029288,MDEyOklzc3VlQ29tbWVudDU4MDAyOTI4OA==,9599,2020-01-30T00:32:43Z,2020-01-30T00:32:43Z,OWNER,"Can you share how your file layout is working?

You should have something like this:

`static/app.css` - a CSS file

Then run Datasette like this:

`datasette my.db --static-dir=static:static/`

Then `http://127.0.0.1:8001/static/app.css` should serve your CSS.

Could you share the command you're using to deploy to Heroku?","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",550293770,
https://github.com/simonw/datasette/issues/577#issuecomment-581758728,https://api.github.com/repos/simonw/datasette/issues/577,581758728,MDEyOklzc3VlQ29tbWVudDU4MTc1ODcyOA==,9599,2020-02-04T06:11:53Z,2020-02-04T06:11:53Z,OWNER,"For the moment I'm going to move it to `async def render_template()` on `datasette` but otherwise keep the implementation the same.

The new signature will be:

    async def render_template(self, template, context=None, request=None, view_name=None):

`template` can be a list of strings or a single string. If a list of strings a template will be selected from them.

I'll reconsider the large list of default context variables later on in a separate ticket.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",497171390,
https://github.com/simonw/datasette/issues/675#issuecomment-589908912,https://api.github.com/repos/simonw/datasette/issues/675,589908912,MDEyOklzc3VlQ29tbWVudDU4OTkwODkxMg==,9599,2020-02-22T02:38:21Z,2020-02-22T02:38:21Z,OWNER,"Interesting feature suggestion.

My initial instinct was that this would be better handled using the layered nature of Docker - so build a Docker image with `datasette package` and then have a separate custom script which takes that image, copies in the extra data and outputs a new image.

But... `datasette package` is already meant to be more convenient than messing around with Docker by hand like this - so actually having a `--copy` option like you describe here feels like it's within scope of what `datasette package` is meant to do.

So yeah - if you're happy to design this I think it would be worth us adding.

Small design suggestion: allow `--copy` to be applied multiple times, so you can do something like this:

    datasette package \
        --copy ~/project/templates /templates \
        --copy ~/project/README.md /README.md \
        data.db

Also since Click arguments can take multiple options I don't think you need to have the `:` in there - although if it better matches Docker's own UI it might be more consistent to have it.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",567902704,
https://github.com/simonw/datasette/issues/682#issuecomment-590517338,https://api.github.com/repos/simonw/datasette/issues/682,590517338,MDEyOklzc3VlQ29tbWVudDU5MDUxNzMzOA==,9599,2020-02-24T19:51:21Z,2020-02-24T19:51:21Z,OWNER,"I filed a question / feature request with Janus about supporting timeouts for `.get()` against async queues here: https://github.com/aio-libs/janus/issues/240

I'm going to move ahead without needing that ability though. I figure SQLite writes are _fast_, and plugins can be trusted to implement just fast writes. So I'm going to support either fire-and-forget writes (they get added to the queue and a task ID is returned) or have the option to block awaiting the completion of the write (using Janus) but let callers decide which version they want. I may add optional timeouts some time in the future.

I am going to make both `execute_write()` and `execute_write_fn()` awaitable functions though, for consistency with `.execute()` and to give me flexibility to change how they work in the future.

I'll also add a `block=True` option to both of them which causes the function to wait for the write to be successfully executed - defaults to `False` (fire-and-forget mode).
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",569613563,
https://github.com/simonw/datasette/pull/683#issuecomment-590679273,https://api.github.com/repos/simonw/datasette/issues/683,590679273,MDEyOklzc3VlQ29tbWVudDU5MDY3OTI3Mw==,9599,2020-02-25T04:37:21Z,2020-02-25T04:37:21Z,OWNER,I'm happy with this now. I'm going to merge to master.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",570101428,
https://github.com/simonw/datasette/issues/675#issuecomment-592399256,https://api.github.com/repos/simonw/datasette/issues/675,592399256,MDEyOklzc3VlQ29tbWVudDU5MjM5OTI1Ng==,9599,2020-02-28T08:09:12Z,2020-02-28T08:09:12Z,OWNER,"Sure, `--cp` looks good to me.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",567902704,
https://github.com/simonw/datasette/issues/394#issuecomment-602907207,https://api.github.com/repos/simonw/datasette/issues/394,602907207,MDEyOklzc3VlQ29tbWVudDYwMjkwNzIwNw==,127565,2020-03-23T23:12:18Z,2020-03-23T23:12:18Z,CONTRIBUTOR,"This would also be useful for running Datasette in Jupyter notebooks on [Binder](https://mybinder.org/). While you can use [Jupyter-server-proxy](https://github.com/jupyterhub/jupyter-server-proxy) to access Datasette on Binder, the links are broken.

Why run Datasette on Binder? I'm developing a [range of Jupyter notebooks](https://glam-workbench.github.io/) that are aimed at getting humanities researchers to explore data from libraries, archives, and museums. Many of them are aimed at researchers with limited digital skills, so being able to run examples in Binder without them installing anything is fantastic.

For example, there are a [series of notebooks](https://glam-workbench.github.io/trove-harvester/) that help researchers harvest digitised historical newspaper articles from Trove. The metadata from this harvest is saved as a CSV file that users can download. I've also provided some extra notebooks that use Pandas etc to demonstrate ways of analysing and visualising the harvested data.

But it would be really nice if, after completing a harvest, the user could spin up Datasette for some initial exploration of their harvested data without ever leaving their browser.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",396212021,
https://github.com/simonw/datasette/issues/394#issuecomment-603631640,https://api.github.com/repos/simonw/datasette/issues/394,603631640,MDEyOklzc3VlQ29tbWVudDYwMzYzMTY0MA==,9599,2020-03-25T04:19:08Z,2020-03-25T04:19:08Z,OWNER,Shipped in 0.39: https://datasette.readthedocs.io/en/latest/changelog.html#v0-39,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",396212021,
https://github.com/simonw/datasette/issues/717#issuecomment-610076073,https://api.github.com/repos/simonw/datasette/issues/717,610076073,MDEyOklzc3VlQ29tbWVudDYxMDA3NjA3Mw==,9599,2020-04-06T22:47:21Z,2020-04-06T22:47:21Z,OWNER,I'm confident it's possible to create a plugin that deploys to Now v2 now. I'll do the rest of the work in a separate repo: https://github.com/simonw/datasette-publish-now,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",594189527,
https://github.com/simonw/datasette/issues/176#issuecomment-617208503,https://api.github.com/repos/simonw/datasette/issues/176,617208503,MDEyOklzc3VlQ29tbWVudDYxNzIwODUwMw==,12976,2020-04-21T14:16:24Z,2020-04-21T14:16:24Z,NONE,"@eads I'm interested in helping, if there's still a need...","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",285168503,
https://github.com/simonw/datasette/issues/731#issuecomment-618155472,https://api.github.com/repos/simonw/datasette/issues/731,618155472,MDEyOklzc3VlQ29tbWVudDYxODE1NTQ3Mg==,9599,2020-04-23T03:28:42Z,2020-04-23T03:28:56Z,OWNER,"As an alternative to `--static` this could work by letting you create the following:

- `static/css/`
- `static/js/`

Which would be automatically mounted at `/js/...` and `/css/...`

Or maybe just mount `static/` at `/static/` instead? ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",605110015,
https://github.com/dogsheep/github-to-sqlite/issues/34#issuecomment-622133298,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34,622133298,MDEyOklzc3VlQ29tbWVudDYyMjEzMzI5OA==,9599,2020-04-30T21:48:24Z,2020-04-30T21:48:24Z,MEMBER,"Unfortunately it's not available through any GitHub API - I managed to figure out how to get dependencies, but I need dependents. https://github.com/simonw/til/blob/master/github/dependencies-graphql-api.md","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",610408908,
https://github.com/dogsheep/github-to-sqlite/issues/33#issuecomment-622279374,https://api.github.com/repos/dogsheep/github-to-sqlite/issues/33,622279374,MDEyOklzc3VlQ29tbWVudDYyMjI3OTM3NA==,2029,2020-05-01T07:12:47Z,2020-05-01T07:12:47Z,NONE,"I also go it working with:

```yaml
run: echo ${{ secrets.github_token }} | github-to-sqlite auth
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",609950090,
https://github.com/dogsheep/dogsheep-photos/issues/16#issuecomment-623807568,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/16,623807568,MDEyOklzc3VlQ29tbWVudDYyMzgwNzU2OA==,9599,2020-05-05T02:56:06Z,2020-05-05T02:56:06Z,MEMBER,"I'm pretty sure this is what I'm after. The `groups` table has what looks like identified labels in the rows with category = 2025:

<img width=""1122"" alt=""words__groups__2_528_rows_where_where_category___2025"" src=""https://user-images.githubusercontent.com/9599/81031361-e0df4080-8e40-11ea-9060-6d850aa52140.png"">

Then there's a `ga` table that maps groups to assets:

<img width=""304"" alt=""words__ga__633_653_rows"" src=""https://user-images.githubusercontent.com/9599/81031387-f48aa700-8e40-11ea-9a3d-da23903be928.png"">

And an `assets` table which looks like it has one row for every one of my photos:

<img width=""645"" alt=""words__assets__40_419_rows"" src=""https://user-images.githubusercontent.com/9599/81031402-04a28680-8e41-11ea-8047-e9199d068563.png"">

One major challenge: these UUIDs are split into two integer numbers, `uuid_0` and `uuid_1` - but the main photos database uses regular UUIDs like this:

![image](https://user-images.githubusercontent.com/9599/81031481-39164280-8e41-11ea-983b-005ced641a18.png)

I need to figure out how to match up these two different UUID representations. I asked on Twitter if anyone has any ideas: https://twitter.com/simonw/status/1257500689019703296","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612287234,
https://github.com/simonw/datasette/issues/758#issuecomment-624797119,https://api.github.com/repos/simonw/datasette/issues/758,624797119,MDEyOklzc3VlQ29tbWVudDYyNDc5NzExOQ==,9599,2020-05-06T17:53:46Z,2020-05-06T17:53:46Z,OWNER,It's interesting to hear from someone who's using this feature - I'm considering moving it out into a plugin #647.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612382643,
https://github.com/simonw/datasette/issues/757#issuecomment-624821090,https://api.github.com/repos/simonw/datasette/issues/757,624821090,MDEyOklzc3VlQ29tbWVudDYyNDgyMTA5MA==,9599,2020-05-06T18:41:29Z,2020-05-06T18:41:29Z,OWNER,"OK, I just released 0.41 with that and a bunch of other stuff: https://datasette.readthedocs.io/en/latest/changelog.html#v0-41","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612378203,
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395209,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626395209,MDEyOklzc3VlQ29tbWVudDYyNjM5NTIwOQ==,9599,2020-05-10T21:52:42Z,2020-05-10T21:52:42Z,MEMBER,"Aha! It looks like I accidentally installed the old bplist into the same environment:
```
$ pip freeze | grep bpylist
bpylist==0.1.4
bpylist2==3.0.0
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,
https://github.com/dogsheep/dogsheep-photos/issues/21#issuecomment-626395781,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21,626395781,MDEyOklzc3VlQ29tbWVudDYyNjM5NTc4MQ==,9599,2020-05-10T21:57:09Z,2020-05-10T21:57:09Z,MEMBER,"Yes, I just recreated my virtual environment from scratch and the error went away.

The problem occurred when I ran `pip install datasette-bplist` in the same virtual environment - https://github.com/simonw/datasette-bplist/blob/master/setup.py depends on `bpylist` which is incompatible with `bpylist2`.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",615474990,
https://github.com/simonw/datasette/pull/868#issuecomment-650600606,https://api.github.com/repos/simonw/datasette/issues/868,650600606,MDEyOklzc3VlQ29tbWVudDY1MDYwMDYwNg==,9599,2020-06-27T18:44:28Z,2020-06-27T18:44:28Z,OWNER,"This is really exciting! Thanks so much for looking into this.

I'm interested in moving CI for this repo over to GitHub Actions, so I'd be fine with you getting this to work as an Action rather than through Travis. If you can get it working in Travis though I'll happily land that and figure out how to convert that to GitHub Actions later on.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",646448486,
https://github.com/simonw/datasette/issues/877#issuecomment-652520496,https://api.github.com/repos/simonw/datasette/issues/877,652520496,MDEyOklzc3VlQ29tbWVudDY1MjUyMDQ5Ng==,9599,2020-07-01T16:26:52Z,2020-07-01T16:26:52Z,OWNER,Tokens get verified by plugins. So far there's only one: https://github.com/simonw/datasette-auth-tokens - which has you hard-coding plugins in a configuration file. I have a issue there to add support for database-backed tokens too: https://github.com/simonw/datasette-auth-tokens/issues/1,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",648421105,
https://github.com/simonw/sqlite-utils/issues/121#issuecomment-655652679,https://api.github.com/repos/simonw/sqlite-utils/issues/121,655652679,MDEyOklzc3VlQ29tbWVudDY1NTY1MjY3OQ==,79913,2020-07-08T17:24:46Z,2020-07-08T17:24:46Z,CONTRIBUTOR,"Better transaction handling would be really great. Some of my thoughts on implementing better transaction discipline are in https://github.com/simonw/sqlite-utils/pull/118#issuecomment-655239728.

My preferences:

- Each CLI command should operate in a single transaction so that either the whole thing succeeds or the whole thing is rolled back. This avoids partially completed operations when an error occurs part way through processing. Partially completed operations are typically much harder to recovery from gracefully and may cause inconsistent data states.

- The Python API should be transaction-agnostic and rely on the caller to coordinate transactions. Only the caller knows how individual insert, create, update, etc operations/methods should be bundled conceptually into transactions. When the caller is the CLI, for example, that bundling would be at the CLI command-level. Other callers might want to break up operations into multiple transactions. Transactions are usually most useful when controlled at the application-level (like logging configuration) instead of the library level. The library needs to provide an API that's conducive to transaction use, though.

- The Python API should provide a context manager to provide consistent transactions handling with more useful defaults than Python's `sqlite3` module. The latter issues implicit `BEGIN` statements by default for most DML (`INSERT`, `UPDATE`, `DELETE`, … but not `SELECT`, I believe), but **not** DDL (`CREATE TABLE`, `DROP TABLE`, `CREATE VIEW`, …). Notably, the `sqlite3` module doesn't issue the implicit `BEGIN` until the first DML statement. It _does not_ issue it when entering the `with conn` block, like other DBAPI2-compatible modules do. The `with conn` block for `sqlite3` only arranges to commit or rollback an existing transaction when exiting. Including DDL and `SELECT`s in transactions is important for operation consistency, though. There are several existing bugs.python.org tickets about this and future changes are in the works, but sqlite-utils can provide its own API sooner. sqlite-utils's `Database` class could itself be a context manager (built on the `sqlite3` connection context manager) which additionally issues an explicit `BEGIN` when entering. This would then let Python API callers do something like:

```python
db = sqlite_utils.Database(path)

with db: # ← BEGIN issued here by Database.__enter__
    db.insert(…)
    db.create_view(…)
# ← COMMIT/ROLLBACK issue here by sqlite3.connection.__exit__
```","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",652961907,