id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
1823352380,PR_kwDOBm6k_c5Wfgd9,2118,New JSON design for query views,9599,closed,0,,9700784,11,2023-07-26T23:29:21Z,2023-08-08T01:47:40Z,2023-08-08T01:47:39Z,OWNER,simonw/datasette/pulls/2118,"WIP. Refs:
- #2109 

<!-- readthedocs-preview datasette start -->
----
:books: Documentation preview :books:: https://datasette--2118.org.readthedocs.build/en/2118/

<!-- readthedocs-preview datasette end -->",107914493,pull,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2118/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",0,
317001500,MDU6SXNzdWUzMTcwMDE1MDA=,236,datasette publish lambda plugin,9599,open,0,,,11,2018-04-23T22:10:30Z,2023-03-12T14:04:15Z,,OWNER,,"Refs #217 - create a publish plugin that can deploy to AWS Lambda.

https://docs.aws.amazon.com/lambda/latest/dg/limits.html says lambda packages can be up to 50 MB, so this would only work with smaller databases (the command can check the filesize before attempting to package and deploy it).

Lambdas do get a 512 MB `/tmp` directory too, so for larger databases the function could start and then download up to 512MB from an S3 bucket - so the plugin could take an optional S3 bucket to write to and know how to upload the `.db` file there and then have the lambda download it on startup.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/236/reactions"", ""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
1428630253,I_kwDOBm6k_c5VJyrt,1873,Ensure insert API has good tests for rowid and compound primark key tables,9599,open,0,,8755003,11,2022-10-30T06:22:17Z,2022-12-13T05:29:08Z,,OWNER,,"Following:
- #1866

I need to design and implement various edge-cases or primary keys:

- Table without an auto-incrementing primary key
- Table with compound primary keys
- Table with just a `rowid`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1873/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,reopened
1373595927,I_kwDOBm6k_c5R32kX,1809,`prepare_jinja2_environment()` hook should take `datasette` argument,9599,closed,0,,,11,2022-09-14T21:15:46Z,2022-09-17T03:39:05Z,2022-09-17T03:38:33Z,OWNER,,"That plugin hook's current signature is:

https://github.com/simonw/datasette/blob/610425460b519e9c16d386cb81aa081c9d730ef0/datasette/hookspecs.py#L28-L30

As a result in the first alpha release of `datasette-edit-templates` I had to include this horrific hack: https://github.com/simonw/datasette-edit-templates/blob/087f6a6cabc20020f2b0524f11aa3a7836320848/datasette_edit_templates/__init__.py#L72-L75

```python
@hookimpl
def prepare_jinja2_environment(env):
    # TODO: This should ideally take datasette, but that's not an argument yet
    datasette = inspect.currentframe().f_back.f_back.f_back.f_back.f_locals[""self""]
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1809/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1175854982,I_kwDOBm6k_c5GFh-G,1679,Research: how much overhead does the n=1 time limit have?,9599,closed,0,,3268330,11,2022-03-21T19:27:46Z,2022-03-21T21:55:57Z,2022-03-21T21:55:56Z,OWNER,,"https://github.com/simonw/datasette/blob/1a7750eb29fd15dd2eea3b9f6e33028ce441b143/datasette/utils/__init__.py#L181-L200

```python
@contextmanager
def sqlite_timelimit(conn, ms):
    deadline = time.perf_counter() + (ms / 1000)
    # n is the number of SQLite virtual machine instructions that will be
    # executed between each check. It's hard to know what to pick here.
    # After some experimentation, I've decided to go with 1000 by default and
    # 1 for time limits that are less than 50ms
    n = 1000
    if ms < 50:
        n = 1

    def handler():
        if time.perf_counter() >= deadline:
            return 1

    conn.set_progress_handler(handler, n)
    try:
        yield
    finally:
        conn.set_progress_handler(None, n)
```
How often do I set a time limit of 50 or less? How much slower does it go thanks to this code?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1679/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
963528457,MDU6SXNzdWU5NjM1Mjg0NTc=,1425,render_cell() hook should support returning an awaitable,9599,closed,0,,,11,2021-08-08T22:32:29Z,2021-08-09T07:14:35Z,2021-08-09T03:00:37Z,OWNER,,"Many of the plugin hooks can return an awaitable - e.g. https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hook-extra-template-vars - but `render_cell()` doesn't support this.

I recently found myself wanting to execute an additional SQL query from that hook, but it wasn't possible to do that since I couldn't use `await`.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1425/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
811367257,MDU6SXNzdWU4MTEzNjcyNTc=,1231,Race condition errors in new refresh_schemas() mechanism,9599,closed,0,,,11,2021-02-18T18:49:54Z,2021-07-16T19:44:59Z,2021-07-16T19:44:59Z,OWNER,,I tried running a Locust load test against Datasette and hit an error message about a failure to create tables because they already existed. I think this means there are race conditions in the new `refresh_schemas()` mechanism added in #1150.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1231/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
770448622,MDU6SXNzdWU3NzA0NDg2MjI=,1151,Database class mechanism for cross-connection in-memory databases,9599,closed,0,,6346396,11,2020-12-17T23:25:43Z,2021-01-26T19:07:44Z,2020-12-18T01:01:26Z,OWNER,,"> Next challenge: figure out how to use the `Database` class from https://github.com/simonw/datasette/blob/0.53/datasette/database.py for an in-memory database which persists data for the duration of the lifetime of the server, and allows access to that in-memory database from multiple threads in a way that lets them see each other's changes.

_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1150#issuecomment-747768112_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1151/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
573755726,MDU6SXNzdWU1NzM3NTU3MjY=,690,Mechanism for plugins to add action menu items for various things,9599,closed,0,,6026070,11,2020-03-02T06:48:36Z,2020-10-30T05:20:43Z,2020-10-30T05:20:42Z,OWNER,,"Now that we have support for plugins that can write I'm seeing all sorts of places where a plugin might need to add UI to the table page.

Some examples:

- `datasette-configure-fts` needs to add a ""configure search for this table"" link
- a plugin that lets you render or delete tables needs to add a link or button somewhere
- existing plugins like `datasette-vega` and `datasette-cluster-map` already do this with JavaScript

The challenge here is that multiple plugins may want to do this, so simply overriding templates and populating names blocks doesn't entirely work as templates may override each other.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/690/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
663228985,MDU6SXNzdWU2NjMyMjg5ODU=,904,"datasette.urls.table() / .instance() / .database() methods for constructing URLs, also exposed to templates",9599,closed,0,,6026070,11,2020-07-21T18:42:52Z,2020-10-23T19:44:05Z,2020-10-20T00:51:51Z,OWNER,,"I tried using this block of template in a plugin and got an error:
```html
{% block nav %}
    <p class=""crumbs"">
        <a href=""{{ base_url }}"">home</a> /
        <a href=""{{ database_url(database) }}"">{{ database }}</a> /
        <a href=""{{ database_url(database) }}/{{ table|quote_plus }}"">{{ table }}</a>
    </p>
    {{ super() }}
{% endblock %}
```
Error: `'database_url' is undefined`

That's because `database_url` is only made available by the BaseView template here:

https://github.com/simonw/datasette/blob/d6e03b04302a0852e7133dc030eab50177c37be7/datasette/views/base.py#L110-L125",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/904/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
702069429,MDU6SXNzdWU3MDIwNjk0Mjk=,967,Writable canned queries with magic parameters fail if POST body is empty,9599,closed,0,,,11,2020-09-15T16:14:43Z,2020-09-15T20:13:10Z,2020-09-15T20:13:10Z,OWNER,,"When I try to use the new `?_json=1` feature from #880 with magic parameters from #842 I get this error:

> Incorrect number of bindings supplied. The current statement uses 1, and there are 0 supplied",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/967/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
648637666,MDU6SXNzdWU2NDg2Mzc2NjY=,880,POST to /db/canned-query that returns JSON should be supported (for API clients),9599,closed,0,,5818042,11,2020-07-01T03:14:43Z,2020-09-14T21:28:21Z,2020-09-14T21:25:01Z,OWNER,,Now that CSRF is solved for API requests (#835) it would be good to support API requests to the `.json` extension.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/880/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
632724154,MDU6SXNzdWU2MzI3MjQxNTQ=,805,Writable canned queries live demo on Glitch,9599,open,0,,,11,2020-06-06T20:52:13Z,2020-07-01T22:44:01Z,,OWNER,,"Needs to run somewhere with a mutable disk drive, so not Cloud Run or Heroku or Vercel.

I think I'll put it on Glitch.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/805/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,
572896293,MDU6SXNzdWU1NzI4OTYyOTM=,687,Expand plugins documentation to multiple pages,9599,closed,0,,5533512,11,2020-02-28T17:26:21Z,2020-06-22T03:55:20Z,2020-06-22T03:53:54Z,OWNER,,"I think the plugins docs need to extend beyond a single page now. I want to add a whole section on writing tests for plugins, showing how `httpx` can be used as seen in https://github.com/simonw/datasette-atom/issues/3 and suchlike.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/687/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
628025100,MDU6SXNzdWU2MjgwMjUxMDA=,785,Datasette secret mechanism - initially for signed cookies,9599,closed,0,,5512395,11,2020-05-31T19:14:52Z,2020-06-06T00:43:40Z,2020-06-01T00:18:40Z,OWNER,,"See comment in https://github.com/simonw/datasette/issues/784#issuecomment-636514974

Datasette needs to be able to set signed cookies - which means it needs a mechanism for safely handling a signing secret.

Since Datasette is a long-running process the default behaviour here can be to create a random secret on startup. This means that if the server restarts any signed cookies will be invalidated.

If the user wants a persistent secret they'll have to generate it themselves - maybe by setting an environment variable?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/785/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
565518772,MDU6SXNzdWU1NjU1MTg3NzI=,673,Mechanism for checking if a SQLite database file is safe to open,9599,closed,0,,,11,2020-02-14T19:36:04Z,2020-02-14T20:13:59Z,2020-02-14T20:13:59Z,OWNER,,"Opening a SpatiaLite database file without SpatiaLite will result in errors later on. Same for database files which use custom extensions, like the Apple Photos database.

I've figured out how to tell if a database is safe to open or not:
```sql
select sql from sqlite_master where sql like 'CREATE VIRTUAL TABLE%';
```
This returns the SQL definitions for virtual tables. The bit after `using` tells you what they need.

Run this against a SpatiaLite database and you get the following:
```sql
CREATE VIRTUAL TABLE SpatialIndex USING VirtualSpatialIndex()
CREATE VIRTUAL TABLE ElementaryGeometries USING VirtualElementary()
```
Run it against an Apple Photos `photos.db` file (found with `find ~/Library | grep photos.db`) and you get this (partial list):
```sql
CREATE VIRTUAL TABLE RidList_VirtualReader using RidList_VirtualReaderModule
CREATE VIRTUAL TABLE Array_VirtualReader using Array_VirtualReaderModule
CREATE VIRTUAL TABLE LiGlobals_VirtualBufferReader using VirtualBufferReaderModule
CREATE VIRTUAL TABLE RKPlace_RTree using rtree (modelId,minLongitude,maxLongitude,minLatitude,maxLatitude)
```
For a database with FTS4 you get:
```sql
CREATE VIRTUAL TABLE ""docs_fts"" USING FTS4 (
    [title], [content], content=""docs""
)
```
FTS5:
```sql
CREATE VIRTUAL TABLE [FARA_All_Registrants_fts] USING FTS5 (
                [Name], [Address_1], [Address_2],
                content=[FARA_All_Registrants]
            )
```
So I can use this to figure out all of the `using` pieces and then compare them to a list of known support ones.

_Originally posted by @simonw in https://github.com/simonw/datasette/pull/672#issuecomment-586441484_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/673/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
316444720,MDU6SXNzdWUzMTY0NDQ3MjA=,233,Option to expose expanded foreign keys in JSON/CSV,9599,closed,0,,,11,2018-04-21T00:18:25Z,2018-06-16T22:26:21Z,2018-06-16T22:20:14Z,OWNER,,"https://datasette-cluster-map-demo.datasettes.com/sf-trees-02c8ef1/Street_Tree_List?qCareAssistant=1

![f36b87c0-478e-4d55-9a5f-ad37df0b47cb](https://user-images.githubusercontent.com/9599/39078411-bb3e4f88-44be-11e8-9d0c-d22324793c77.png)

It would be nice if the info bubbles there could expose more than just the IDs, and if the title showed the expanded name of the selected qCareAssistant.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/233/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
268470572,MDU6SXNzdWUyNjg0NzA1NzI=,40,Implement command-line tool interface,9599,closed,0,,2857392,11,2017-10-25T16:47:15Z,2017-11-11T07:27:33Z,2017-11-11T07:27:33Z,OWNER,,"The first version needs to take one or more file names or URLs, then generate and deploy an app to Now. It will assume you already have the now command installed and configured.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/40/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed