issues

Table actions

213 rows where comments = 0, state = "closed" and type = "issue" sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: user, milestone, author_association, created_at (date), updated_at (date), closed_at (date)

type 1

  • issue · 213 ✖

state 1

  • closed · 213 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1553425465 I_kwDOCGYnMM5cl2Q5 522 Add COLUMN_TYPE_MAPPING for timedelta maport 81377 closed 0     0 2023-01-23T16:49:54Z 2023-11-04T00:49:51Z 2023-11-04T00:49:51Z NONE  

Currently trying to create a column with Python type datetime.timedelta results in an error:

```

from sqlite_utils import Database db = Database("test.db") test_tbl = db['test'] test_tbl.insert({'col1': datetime.timedelta()}) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 2979, in insert return self.insert_all( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 3082, in insert_all self.create( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 1574, in create self.db.create_table( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 961, in create_table sql = self.create_table_sql( File "/usr/local/lib/python3.10/dist-packages/sqlite_utils/db.py", line 852, in create_table_sql column_type=COLUMN_TYPE_MAPPING[column_type], KeyError: <class 'datetime.timedelta'> ```

The reason this would be useful is that MySQLdb uses timedelta for MySQL TIME columns:

```

import MySQLdb conn = MySQLdb.connect(host='database', user='user', passwd='pw') csr = conn.cursor() csr.execute("SELECT CAST('11:20' AS TIME)") tuple(csr) ((datetime.timedelta(seconds=40800),),) ```

So currently any attempt to convert a MySQL DB with a TIME column using db-to-sqlite will result in the above error.

I was rather surprised that MySQLdb uses timedelta for TIME columns but I see that this column type is intended for time intervals as well as the time of day so it makes sense.

sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/522/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1886649402 I_kwDOBm6k_c5wc_w6 2179 Flaky test: test_hidden_sqlite_stat1_table simonw 9599 closed 0     0 2023-09-07T22:48:43Z 2023-09-07T22:51:19Z 2023-09-07T22:51:19Z OWNER  

This test here: https://github.com/simonw/datasette/blob/fbcb103c0cb6668018ace539a01a6a1f156e8d6a/tests/test_api.py#L1011-L1020

It failed for me like this:

E AssertionError: assert [('normal', False), ('sqlite_stat1', True), ('sqlite_stat4', True)] in ([('normal', False), ('sqlite_stat1', True)],)

Looks like some builds of SQLite include a sqlite_stat4 table.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2179/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1876407598 I_kwDOBm6k_c5v17Uu 2169 execute-sql on a database should imply view-database/view-permission simonw 9599 closed 0     0 2023-08-31T22:45:56Z 2023-08-31T22:46:28Z 2023-08-31T22:46:28Z OWNER  

I noticed that a token with execute-sql permission alone did not work, because it was not allowed to view the instance of the database.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2169/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1871935751 I_kwDOD079W85vk3kH 40 ImportError: cannot import name 'formatargspec' from 'inspect' hosslikw 36752421 closed 0     0 2023-08-29T15:36:31Z 2023-08-31T03:18:07Z 2023-08-31T03:18:06Z NONE  

I get the following error when running "pip3 install dogsheep-photos" " from inspect import ismethod, isclass, formatargspec ImportError: cannot import name 'formatargspec' from 'inspect' (/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/inspect.py). Did you mean: 'formatargvalues'?"

Python 3.12.0rc1 sqlite 3.43.0 datasette, version 0.64.3

 dogsheep-photos 256834907 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/40/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1795051447 I_kwDOBm6k_c5q_k-3 2097 Drop Python 3.7 simonw 9599 closed 0     0 2023-07-08T18:39:44Z 2023-08-23T18:18:00Z 2023-08-23T18:18:00Z OWNER  

I'm going to drop Python 3.7.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1153#issuecomment-1627455892

It's not supported any more: https://devguide.python.org/versions/

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2097/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1822940964 I_kwDOBm6k_c5sp98k 2115 Ensure all tests pass against new query view JSON simonw 9599 closed 0   Datasette 1.0a3 9700784 0 2023-07-26T18:25:20Z 2023-08-08T02:01:39Z 2023-08-08T02:01:38Z OWNER  

  • 2109

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2115/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1786243905 I_kwDOCGYnMM5qd-tB 564 Document that running `db.transform()` tidies up the schema indentation simonw 9599 closed 0     0 2023-07-03T13:59:28Z 2023-07-22T22:15:34Z 2023-07-22T22:15:34Z OWNER  

... and it turns out running .transform() with no arguments still fixes the format of the schema!

```pycon

db["log"].add_column("foo", str)

<Table log (id, name2, age, weight, foo)> >>> db["log"].add_column("bar", str) <Table log (id, name2, age, weight, foo, bar)> >>> db["log"].add_column("baz", str) <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT , [foo] TEXT, [bar] TEXT, [baz] TEXT) >>> db["log"].transform() <Table log (id, name2, age, weight, foo, bar, baz)> >>> print(db["log"].schema) CREATE TABLE "log" ( [id] INTEGER PRIMARY KEY, [name2] TEXT, [age] INTEGER, [weight] FLOAT, [foo] TEXT, [bar] TEXT, [baz] TEXT ) ``` _Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1618347727_
 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/564/reactions",
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 1
}
   completed
1816857105 I_kwDOCGYnMM5sSwoR 570 `sqlite-utils install -e` option simonw 9599 closed 0     0 2023-07-22T18:32:23Z 2023-07-22T18:55:59Z 2023-07-22T18:32:56Z OWNER  

As seen in LLM.

Needed while working on: - #567

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/570/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1773450152 I_kwDOCGYnMM5ptLOo 559 sqlean support simonw 9599 closed 0     0 2023-06-25T19:27:26Z 2023-06-25T23:25:53Z 2023-06-25T23:25:53Z OWNER  

If sqlean is available, use that.

Refs: - https://github.com/nalgeon/sqlean.py/issues/1#issuecomment-1605707788

This will provide a good workaround for: - #235

sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/559/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1701018909 I_kwDOCGYnMM5lY30d 543 Tests broken on Windows due to new convert() lambda names simonw 9599 closed 0     0 2023-05-08T22:11:29Z 2023-05-08T22:19:04Z 2023-05-08T22:19:04Z OWNER  

https://github.com/simonw/sqlite-utils/actions/runs/4920084038/jobs/8788501314 python sql = 'update [example] set [dt] = lambda_-9223371942137158589([dt]);' From: - #526

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/543/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1432377191 I_kwDOCGYnMM5VYFdn 509 `sqlite-utils transform` breaks DEFAULT string values and STRFTIME() kennysong 2199875 closed 0     0 2022-11-02T02:32:23Z 2023-05-08T21:13:38Z 2023-05-08T21:13:38Z NONE  

Very nice library! Our team found sqlite-utils through @simonw's comment on the "Simple declarative schema migration for SQLite" article, and we were excited to use it, but unfortunately sqlite-utils transform seems to break our DB.

Running sqlite-utils transform to modify a column mangles their DEFAULT values:

  • Default string values are wrapped in extra single quotes
  • Function expressions such as STRFTIME() are turned into strings!

Here are steps to reproduce:

Original database

``` $ sqlite3 test.db << EOF CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) EOF

$ sqlite3 test.db "SELECT sql FROM sqlite_master WHERE name = 'mytable';" CREATE TABLE mytable ( col1 TEXT DEFAULT 'foo', col2 TEXT DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')) ) ```

Modified database after sqlite-utils

``` $ sqlite3 test.db "INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;" foo|2022-11-02 02:26:58.038

$ sqlite-utils transform test.db mytable --rename col1 renamedcol1

$ sqlite3 test.db "SELECT sql FROM sqlite_master WHERE name = 'mytable';" CREATE TABLE "mytable" ( [renamedcol1] TEXT DEFAULT '''foo''', [col2] TEXT DEFAULT 'STRFTIME(''%Y-%m-%d %H:%M:%f'', ''NOW'')' )

$ sqlite3 test.db "INSERT INTO mytable DEFAULT VALUES; SELECT * FROM mytable;" foo|2022-11-02 02:26:58.038 'foo'|STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW') ```

(Related: #336)

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/509/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1661617056 I_kwDODD6af85jCkOg 15 ambiguous column name: createdAt - on checkin_details view simonw 9599 closed 0     0 2023-04-11T01:07:47Z 2023-04-11T03:16:37Z 2023-04-11T03:16:37Z MEMBER  

It looks like Swarm changed their schema and now both venues and checkins have createdAt fields.

Which breaks this view: https://github.com/dogsheep/swarm-to-sqlite/blob/719b6e96a016d0ca8b316d3bed9c2a7a0cb499ee/swarm_to_sqlite/utils.py#L171-L188

 swarm-to-sqlite 205429375 issue     
{
    "url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/15/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1617962395 I_kwDOJHON9s5gcCWb 10 Include schema in README simonw 9599 closed 0     0 2023-03-09T20:38:59Z 2023-03-09T20:48:18Z 2023-03-09T20:48:18Z MEMBER  

As seen in other tools like https://github.com/simonw/git-history

 apple-notes-to-sqlite 611552758 issue     
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/10/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1616422013 I_kwDOJHON9s5gWKR9 3 `apple-notes-to-sqlite --dump` option simonw 9599 closed 0     0 2023-03-09T05:05:49Z 2023-03-09T05:06:14Z 2023-03-09T05:06:14Z MEMBER  

Option that doesn't write to the database at all, it just outputs all the notes to stdout as newline-delimited JSON.

 apple-notes-to-sqlite 611552758 issue     
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/3/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1515186569 I_kwDOBm6k_c5aT-mJ 1972 Fix Sphinx warning about extlink extension simonw 9599 closed 0     0 2022-12-31T19:12:04Z 2022-12-31T19:13:26Z 2022-12-31T19:13:26Z OWNER  

[sphinx-autobuild] > sphinx-build -b html /Users/simon/Dropbox/Development/datasette/docs /Users/simon/Dropbox/Development/datasette/docs/_build Running Sphinx v5.3.0 loading pickled environment... done WARNING: extlinks: Sphinx-6.0 will require a caption string to contain exactly one '%s' and all other '%' need to be escaped as '%%'.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1971#issuecomment-1368266904

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1972/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1493306655 I_kwDOBm6k_c5ZAg0f 1945 `view-instance` should not be checked for /-/actor.json simonw 9599 closed 0   Datasette 1.0a2 8711695 0 2022-12-13T04:01:46Z 2022-12-13T04:11:56Z 2022-12-13T04:11:56Z OWNER  

Spotted this while testing:

  • 1855

    export TOKEN=$(datasette create-token root --secret s -a foo) curl -H "Authorization: Bearer $TOKEN" http://localhost:8002/-/actor.json Returned a Forbidden error (and not in JSON either).
 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1945/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1487764628 I_kwDOCGYnMM5YrXyU 518 flake8 ValueError: Error code '#' supplied to 'extend-ignore' option... simonw 9599 closed 0     0 2022-12-10T01:30:24Z 2022-12-10T01:36:46Z 2022-12-10T01:36:46Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/518/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1455932972 I_kwDOBm6k_c5Wx8Ys 1904 Datasette Lite tests failing due to httpx upgrade simonw 9599 closed 0   Datasette 1.0a0 8658075 0 2022-11-18T22:49:31Z 2022-11-18T22:57:48Z 2022-11-18T22:52:22Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1904/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1450796965 I_kwDOBm6k_c5WeWel 1894 Initialize CodeMirror during DOMContentLoaded instead of onload bgrins 95570 closed 0     0 2022-11-16T03:52:19Z 2022-11-18T07:29:02Z 2022-11-18T07:29:02Z CONTRIBUTOR  

As per https://github.com/simonw/datasette/pull/1893/files#r1023248927 this should prevent a flash between the textarea being replaced by CodeMirror.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1894/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1400083043 I_kwDOBm6k_c5Tc5Jj 1834 inspect data is not used for caching database hash fgregg 536941 closed 0     0 2022-10-06T17:52:01Z 2022-10-06T20:06:21Z 2022-10-06T20:06:08Z CONTRIBUTOR  

When databases are loaded,

https://github.com/simonw/datasette/blob/cb1e093fd361b758120aefc1a444df02462389a3/datasette/app.py#L257-L260

there is nothing preventing the rehashing of the database for immutable databases.

https://github.com/simonw/datasette/blob/cb1e093fd361b758120aefc1a444df02462389a3/datasette/database.py#L50-L53

what i might expect is that relevant values of inspect_data get passed to the Database class to prevent re-hashing?

With data that is many gigs large, this is a significant start up time.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1834/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1343732788 I_kwDOBm6k_c5QF7w0 1788 Make it more obvious that Datasette publish can publish multiple databases simonw 9599 closed 0     0 2022-08-18T22:57:51Z 2022-08-18T23:06:16Z 2022-08-18T23:06:16Z OWNER  

Feedback initially for datasette-publish-fly but it applies to the others too.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1788/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1292368833 I_kwDOBm6k_c5NB_vB 1764 Keep track of config_dir in directory mode (for plugins) eyeseast 25778 closed 0     0 2022-07-03T16:57:49Z 2022-07-18T01:12:45Z 2022-07-18T01:12:45Z CONTRIBUTOR  

I started working on using config_dir with my datasette-query-files plugin and realized Datasette doesn't actually hold onto the config_dir argument. It gets used in __init__ but then forgotten. It would be nice to be able to use it in plugins, though.

Here's the reference issue: https://github.com/eyeseast/datasette-query-files/issues/4

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1764/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1243715381 I_kwDOCGYnMM5KIZc1 436 Add "copy to clipboard" button to code examples in documentation simonw 9599 closed 0     0 2022-05-20T21:53:23Z 2022-05-20T21:57:53Z 2022-05-20T21:57:53Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/436/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1177059481 I_kwDODFdgUs5GKICZ 71 Store commit parents carltongibson 64686 closed 0     0 2022-03-22T17:06:48Z 2022-04-22T12:44:04Z 2022-04-22T12:44:04Z NONE  

Hi @simonw 👋

Currently, stored commit data doesn't quite give me the information I'm needing...

Committer date and author date are not 100% reliable for dividing a commit history up by release or branch. A PR created before a release but merged after can have earlier dates… — this can be quite frustrating if you're trying to pin down commits for a release: It should be there!, but then isn't. (This gets worse using release branches.)

Would you be open to adding the sha of a parent of a commit to the commit table? (As an FK? 🤔 — likely not feasible.)

It's part of the response body:

"parents": [ { "url": "https://api.github.com/repos/octocat/Hello-World/commits/6dcb09b5b57875f334f61aebed695e2e4193db5e", "sha": "6dcb09b5b57875f334f61aebed695e2e4193db5e" } ],

I think this list should only have a single entry. (🤔 — not sure why it's a list then...)

With this it would be possible to build/reconstruct a chain of commits from the history, that I don't think is available as yet (unless you know a better way).

It is certainly possible to get sequential lists of commits out of git directly, so the same would be possible combining tools, but wondering if a single tool could do it.

What do you think? Thanks! 🏅

github-to-sqlite 207052882 issue     
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/71/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1194790504 I_kwDOBm6k_c5HNw5o 1701 Use + for spaces instead of ~20 simonw 9599 closed 0   Datasette 1.0 3268330 0 2022-04-06T15:40:48Z 2022-04-06T15:55:10Z 2022-04-06T15:55:05Z OWNER  

Tilde encoding introduced in #1657 means that database files with spaces in the name - e.g. the Apple Mail Envelope Index database - end up with URLs like this:

http://127.0.0.1:8001/Envelope~20Index

I think this would be prettier:

http://127.0.0.1:9933/Envelope+Index
datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1701/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1182143895 I_kwDOBm6k_c5GdhWX 1691 Bug in pytest-httpx example simonw 9599 closed 0     0 2022-03-26T22:45:30Z 2022-03-26T22:46:09Z 2022-03-26T22:46:09Z OWNER  

https://docs.datasette.io/en/0.61.1/testing_plugins.html#testing-outbound-http-calls-with-pytest-httpx says:

python async def test_outbound_http_call(httpx_mock): httpx_mock.add_response( url='https://www.example.com/', data='Hello world', ) That's wrong - data= should be text=.

https://github.com/Colin-b/pytest_httpx/blob/v0.20.0/README.md#reply-with-custom-body

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1691/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1178456794 I_kwDOCGYnMM5GPdLa 418 Add generated files to .gitignore eyeseast 25778 closed 0     0 2022-03-23T17:48:12Z 2022-03-24T21:01:44Z 2022-03-24T21:01:44Z CONTRIBUTOR  

I end up with these in my local directory:

.hypothesis/
Pipfile
Pipfile.lock
pyproject.toml

Might as well gitignore them.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/418/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1179928510 I_kwDOBm6k_c5GVEe- 1683 allow_facet: False should be respected by column cog menu simonw 9599 closed 0     0 2022-03-24T19:05:06Z 2022-03-24T19:16:36Z 2022-03-24T19:16:36Z OWNER  

The column cog menu currently shows "Facet by this" even if faceting is disabled for the Datasette instance.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1683/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1122413719 I_kwDOBm6k_c5C5qyX 1621 Test against Python 3.11 dev version simonw 9599 closed 0   Datasette 1.0 3268330 0 2022-02-02T21:38:57Z 2022-03-19T04:04:49Z 2022-02-02T21:58:54Z OWNER  

To avoid another surprise like we got with 3.10: https://simonwillison.net/2021/Oct/9/finding-and-reporting-a-bug/

From a quick GitHub code search it looks like 3.11-dev should work: https://cs.github.com/urllib3/urllib3/blob/7bec77e81aa0a194c98381053225813f5347c9d2/.github/workflows/ci.yml#L60

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1621/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1123851690 I_kwDOCGYnMM5C_J2q 396 mypy failure, sqlite_utils/utils.py:56 simonw 9599 closed 0     0 2022-02-04T06:08:09Z 2022-02-04T06:10:33Z 2022-02-04T06:10:33Z OWNER  

https://github.com/simonw/sqlite-utils/runs/5062725880?check_suite_focus=true

sqlite_utils/utils.py:56: error: Incompatible return value type (got "None", expected "str")

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/396/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1099897648 I_kwDOCGYnMM5Bjxsw 384 Add examples to every `--help` simonw 9599 closed 0     0 2022-01-12T05:31:25Z 2022-01-26T03:15:02Z 2022-01-26T03:15:02Z OWNER  

Everything on https://sqlite-utils.datasette.io/en/stable/cli-reference.html would benefit from an example.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/384/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1107557831 I_kwDOCGYnMM5CA_3H 386 Better "contributing" documentation simonw 9599 closed 0     0 2022-01-19T02:11:48Z 2022-01-19T02:15:21Z 2022-01-19T02:15:21Z OWNER  

This page jumps straight into running the tests: https://sqlite-utils.datasette.io/en/latest/contributing.html

It should add a little more about expected collaboration styles - opening an issue before filing a pull request - and probably link to https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/386/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1082746149 I_kwDOBm6k_c5AiWUl 1560 Table page title has "where where" in it simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-17T00:05:48Z 2022-01-13T22:28:35Z 2022-01-13T22:20:15Z OWNER  

Just noticed this while working on #1518.

% curl -s 'https://latest.datasette.io/fixtures/facetable?_sort=pk&on_earth__exact=1' | grep -C 1 '<title>' <head> <title>fixtures: facetable: 14 rows where where on_earth = 1 sorted by pk</title>

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1560/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083927147 I_kwDOBm6k_c5Am2pr 1571 Track number of executions for execute_write_many() in traces simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-18T19:16:17Z 2022-01-13T22:27:49Z 2021-12-19T20:30:40Z OWNER  

Spotted while working on #1555

There's no indication there of how many times execute_write_many() executed the SQL.

Solving this is a tiny bit tricky because params_seq is an iterator that we don't want to exhaust before passing it to conn.executemany() - so we need to instead wrap it in something that counts how many times it was called.

But then we need a way to attach that to the trace here: https://github.com/simonw/datasette/blob/d637ed46762fdbbd8e32b86f258cd9a53c1cfdc7/datasette/database.py#L115-L122

So probably need to redesign the trace() decorator to allow extra pairs to be attached to it within the with statement.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1571/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1084007781 I_kwDOBm6k_c5AnKVl 1572 "Query took" should be "Queries took" simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-19T04:03:00Z 2022-01-13T22:27:43Z 2021-12-19T04:03:24Z OWNER  

This is misleading, since usually there have been more than one query executed:

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1572/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083718998 I_kwDOBm6k_c5AmD1W 1567 Remove undocumented sqlite_functions mechanism simonw 9599 closed 0   Datasette 0.60 7571612 0 2021-12-18T01:51:10Z 2022-01-13T22:27:04Z 2021-12-18T01:54:46Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1567/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1098544628 I_kwDOCGYnMM5BenX0 379 CLI options for running ANALYZE simonw 9599 closed 0   3.21 7558727 0 2022-01-11T01:09:16Z 2022-01-11T01:38:01Z 2022-01-11T01:36:48Z OWNER  

The Python methods are all done now, next step is the CLI options. I'll do those in a separate issue.

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009508865

  • [x] sqlite-utils analyze command
  • [x] sqlite-utils create-index --analyze option (see #365)
  • [x] sqlite-utils insert --analyze option
  • [x] sqlite-utils upsert --analyze option

In #378 I also added .delete_where(..., analyze=True) but there isn't currently a sqlite-utils delete-where CLI command - deletions via CLI are expected to be handled using SQL queries.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/379/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1098309897 I_kwDOCGYnMM5BduEJ 378 analyze=True parameter for some methods simonw 9599 closed 0   3.21 7558727 0 2022-01-10T19:54:52Z 2022-01-11T01:08:11Z 2022-01-11T01:08:09Z OWNER  

This would cause ANALYZE to be run against the relevant table at the end of executing the method.

Having browsed the API reference I think the methods that would benefit from an analyze=True parameter are:

  • [x] table.create_index
  • [x] table.insert_all
  • [x] table.upsert_all
  • [x] table.delete_where

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/366#issuecomment-1009288898

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/378/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1097436959 I_kwDOCGYnMM5BaY8f 376 `--nl` mode should ignore blank lines simonw 9599 closed 0   3.21 7558727 0 2022-01-10T04:10:54Z 2022-01-10T19:27:41Z 2022-01-10T04:12:46Z OWNER  

Spotted this while manually testing #364 - there's no reason --nl should crash if you feed it an empty line in between JSON objects.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/376/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1094974713 I_kwDOCGYnMM5BQ_z5 362 upsert --detect-types is broken simonw 9599 closed 0     0 2022-01-06T05:12:10Z 2022-01-06T06:54:45Z 2022-01-06T06:28:34Z OWNER  

Noticed this thanks to syntax highlighting in VS Code showing an unused variable - need to fix it and add a test.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/362/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1053136495 I_kwDOCGYnMM4-xZZv 341 `hash_id: Optional[Any]` should be `hash_id: Optional[str]` simonw 9599 closed 0     0 2021-11-15T02:12:39Z 2021-11-15T02:19:31Z 2021-11-15T02:19:31Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/341/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1053087862 I_kwDOCGYnMM4-xNh2 338 dict, list, tuple should all map to TEXT simonw 9599 closed 0     0 2021-11-15T00:28:01Z 2021-11-15T00:36:03Z 2021-11-15T00:36:03Z OWNER  

This relates to the fact that dictionaries, lists and tuples get special treatment and are converted to JSON strings, using this code: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L2937-L2947

So the COLUMN_TYPE_MAPPING should include those too - right now it looks like this: https://github.com/simonw/sqlite-utils/blob/e8d958109ee290cfa1b44ef7a39629bb50ab673e/sqlite_utils/db.py#L165-L188

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/322#issuecomment-968401459

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/338/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1041778507 I_kwDOCGYnMM4-GEdL 334 Filter by datetime objects using rows_where() viseshrp 11642379 closed 0     0 2021-11-02T00:44:08Z 2021-11-13T19:23:21Z 2021-11-13T19:23:21Z NONE  

Firstly, thanks for this nice utility. It would be nice to have an example in the docs on how to filter by date range using rows_where(). This doesn't seem to work: table.rows_where('datetime(created) between datetime("2021-10-31T17:29:59.277428-04:00") AND datetime("2021-11-01T03:44:04.544651+00:00")')

I could probably just use db.query(), which works for the above, but it would be nice if I could pass in datetime objects in rows_where(). Thanks.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/334/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
988553806 MDU6SXNzdWU5ODg1NTM4MDY= 1457 suggestion: distinguish names in `--static` documentation ctb 51016 closed 0     0 2021-09-05T17:04:27Z 2021-10-14T18:39:55Z 2021-10-14T18:39:55Z CONTRIBUTOR  

Over in https://docs.datasette.io/en/stable/custom_templates.html#serving-static-files, there is the slightly comical example command -

datasette -m metadata.json --static static:static/ --memory

(now, with MORE STATIC!)

It took me a while to sort out all the URLs and paths involved because I wasn't being very clever. But in the interests of simplification and distinction, I might suggest something like

datasette -m metadata.json --static loc:static-files/ --memory

I will submit a PR for your consideration.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1457/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1023243105 I_kwDOBm6k_c48_XNh 1486 pipx installation instructions for plugins don't reference pipx inject RhetTbull 41546558 closed 0     0 2021-10-12T00:43:42Z 2021-10-13T21:09:11Z 2021-10-13T21:09:11Z CONTRIBUTOR  

The datasette installation instructions discuss how to install with pipx, how to upgrade with pipx, and how to upgrade plugins with pipx but do not mention how to install a plugin with pipx. You discussed this on your blog but looks like this didn't make it in when you updated the docs for pipx (#756).

I'll submit a PR shortly to fix this.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1486/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1021849766 I_kwDOBm6k_c486DCm 1483 Running a search on page 2 of results should not preserve ?_next= simonw 9599 closed 0     0 2021-10-10T01:18:12Z 2021-10-13T21:08:10Z 2021-10-13T21:08:10Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1483/reactions",
    "total_count": 2,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
975158266 MDU6SXNzdWU5NzUxNTgyNjY= 19 table activity_summary has no column named appleMoveTime simonw 9599 closed 0     0 2021-08-20T00:46:44Z 2021-08-20T00:54:34Z 2021-08-20T00:54:34Z MEMBER  

Got this error today against a fresh export:

table activity_summary has no column named appleMoveTime
healthkit-to-sqlite 197882382 issue     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/19/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
931752773 MDU6SXNzdWU5MzE3NTI3NzM= 294 Add a `sqlite-utils memory` example to the README simonw 9599 closed 0     0 2021-06-28T16:35:59Z 2021-08-18T21:40:03Z 2021-08-18T21:40:03Z OWNER  
sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/294/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
965166058 MDU6SXNzdWU5NjUxNjYwNTg= 313 `.add_foreign_keys()` doesn't reject being called with a View simonw 9599 closed 0     0 2021-08-10T17:22:17Z 2021-08-10T17:25:34Z 2021-08-10T17:25:34Z OWNER  

Spotted this bug using mypy while working on #311 / #312!

% mypy sqlite_utils sqlite_utils/db.py:725: error: Item "View" of "Union[Table, View]" has no attribute "foreign_keys" Found 1 error in 1 file (checked 5 source files) Refers to this code: https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L710-L720

It's a bug! We run some checks earlier but none of them ensure that it's a view:

https://github.com/simonw/sqlite-utils/blob/c11ff89894727270d4a9eb554d3a006f5b0d8d9d/sqlite_utils/db.py#L697-L709

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/313/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
959305209 MDU6SXNzdWU5NTkzMDUyMDk= 307 codespell to spell check documentation simonw 9599 closed 0     0 2021-08-03T16:48:19Z 2021-08-03T16:48:53Z 2021-08-03T16:48:53Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/307/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
959276629 MDU6SXNzdWU5NTkyNzY2Mjk= 1416 Use rich to render tracebacks on errors, if available simonw 9599 closed 0     0 2021-08-03T16:12:08Z 2021-08-03T16:12:51Z 2021-08-03T16:12:51Z OWNER  

Now thinking I should try adding Rich as an optional dependency to Datasette - if it's there, show tracebacks using it. Could be really handy for development https://twitter.com/simonw/status/1422576091055616003

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1416/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
957383814 MDU6SXNzdWU5NTczODM4MTQ= 301 insert-files should get a --silent option simonw 9599 closed 0     0 2021-08-01T04:11:03Z 2021-08-02T19:12:21Z 2021-08-02T19:12:21Z OWNER  

The new sqlite-utils convert command I'm adding in #251 will have a --silent option for turning off the progress bars. The only other command that has progress bars right now is insert-files so it should get this option too, for consistency.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/301/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
927789811 MDU6SXNzdWU5Mjc3ODk4MTE= 292 Add contributing documentation simonw 9599 closed 0     0 2021-06-23T02:13:05Z 2021-06-25T17:53:51Z 2021-06-25T17:53:51Z OWNER  

Like https://docs.datasette.io/en/latest/contributing.html (but simpler) - should cover how to run black and flake8 and mypy and how to run the tests.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/292/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
925544070 MDU6SXNzdWU5MjU1NDQwNzA= 287 Update rowid examples in the docs simonw 9599 closed 0     0 2021-06-20T08:03:00Z 2021-06-20T18:26:21Z 2021-06-20T18:26:21Z OWNER  

Changed in #284 - a couple of examples need updating on https://github.com/simonw/sqlite-utils/blob/3.10/docs/cli.rst.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/287/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
925545468 MDU6SXNzdWU5MjU1NDU0Njg= 288 sqlite-utils memory blah.json --schema simonw 9599 closed 0     0 2021-06-20T08:10:40Z 2021-06-20T18:26:21Z 2021-06-20T18:26:21Z OWNER  

Like --dump but only outputs the schema - useful for understanding what you are about to run queries against.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/288/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
924991194 MDU6SXNzdWU5MjQ5OTExOTQ= 280 Add --encoding option to sqlite-utils memory simonw 9599 closed 0     0 2021-06-18T15:03:32Z 2021-06-18T15:29:46Z 2021-06-18T15:29:46Z OWNER  

Follow-on from #272 - this will work like --encoding on sqlite-utils insert and will affect all CSV files processed by sqlite-utils memory.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/280/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
923602693 MDU6SXNzdWU5MjM2MDI2OTM= 276 support small help flag -h mcint 601708 closed 0     0 2021-06-17T07:59:31Z 2021-06-18T14:56:59Z 2021-06-18T14:56:59Z CONTRIBUTOR  
sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/276/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
922832113 MDU6SXNzdWU5MjI4MzIxMTM= 274 sqlite-utils dump my.db command simonw 9599 closed 0     0 2021-06-16T16:30:14Z 2021-06-16T23:51:54Z 2021-06-16T23:51:54Z OWNER  

Inspired by the --dump mechanism I added to sqlite-utils memory here: https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862018937

Can use .iterdump() to implement this: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.iterdump

Maybe instead (or as-well-as) offer --dump which dumps out the SQL from that.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/274/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
919733213 MDU6SXNzdWU5MTk3MzMyMTM= 33 Searching for whitespace throws an error simonw 9599 closed 0     0 2021-06-13T06:57:57Z 2021-06-13T14:36:39Z 2021-06-13T14:36:39Z MEMBER  

https://datasette.io/-/beta?q=+ returns a 500

fts5: syntax error near ""

 dogsheep-beta 197431109 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/33/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
912959264 MDU6SXNzdWU5MTI5NTkyNjQ= 1364 Don't truncate columns on the list of databases simonw 9599 closed 0     0 2021-06-06T22:01:56Z 2021-06-06T22:07:50Z 2021-06-06T22:07:50Z OWNER  

https://covid-19.datasettes.com/covid currently truncates at 9 database columns:

Django SQL Dashboard showed me that this is a bad idea - having the full list of columns is actually really useful documentation for crafting custom SQL queries.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1364/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
912419349 MDU6SXNzdWU5MTI0MTkzNDk= 1359 `?_trace=1` should only be available with a new `trace_debug` setting simonw 9599 closed 0     0 2021-06-05T19:59:27Z 2021-06-05T20:18:46Z 2021-06-05T20:18:46Z OWNER  

Just like template debug mode is controlled by this off-by-default setting: https://github.com/simonw/datasette/blob/368aa5f1b16ca35f82d90ff747023b9a2bfa27c1/datasette/app.py#L160-L164

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1359/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
864969683 MDU6SXNzdWU4NjQ5Njk2ODM= 1305 Index view crashes when any database table is not accessible to actor gfrmin 416374 closed 0     0 2021-04-22T13:44:22Z 2021-06-02T04:26:29Z 2021-06-02T04:26:29Z CONTRIBUTOR  

Because of https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L63, the tables dict built does not include invisible tables; however, if https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L80 is reached (because table_counts was not successfully initialized, e.g. due to a very large database) then as db.get_all_foreign_keys() returns ALL tables, a KeyError will be raised.

This error can be recreated with the fixtures.db if any table is hidden, e.g. by adding something like "foreign_key_references": { "allow": {} } to fixtures-metadata.json and deleting or not table_counts from https://github.com/simonw/datasette/blob/main/datasette/views/index.py#L77.

I'm not sure how to fix this error; perhaps by testing if the table is in the aforementions tables dict.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1305/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
858501079 MDU6SXNzdWU4NTg1MDEwNzk= 255 transform --help should tell you the available types simonw 9599 closed 0     0 2021-04-15T05:24:48Z 2021-05-29T03:55:52Z 2021-05-29T03:55:52Z OWNER  

``` Usage: sqlite-utils transform [OPTIONS] PATH TABLE

Transform a table beyond the capabilities of ALTER TABLE

Options: --type <TEXT TEXT>... Change column type to X ``` This should specify that the possible types are 'INTEGER', 'TEXT', 'FLOAT', 'BLOB'.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/255/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
899169307 MDU6SXNzdWU4OTkxNjkzMDc= 1338 Fix jinja2 warnings simonw 9599 closed 0     0 2021-05-24T01:38:23Z 2021-05-24T01:41:55Z 2021-05-24T01:41:55Z OWNER  

Lots of these in the test suite now, after the Jinja upgrade in #1331: ``` tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:45: DeprecationWarning: 'jinja2.escape' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.escape' instead. label=jinja2.escape(data["label"] or "") or " ",

tests/test_plugins.py::test_hook_render_cell_link_from_json datasette/tests/plugins/my_plugin_2.py:41: DeprecationWarning: 'jinja2.Markup' is deprecated and will be removed in Jinja 3.1. Import 'markupsafe.Markup' instead. return jinja2.Markup( ```

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1338/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
868188068 MDU6SXNzdWU4NjgxODgwNjg= 257 Insert from JSON containing strings with non-ascii characters are escaped as unicode for lists, tuples, dicts. dylan-wu 6586811 closed 0     0 2021-04-26T20:46:25Z 2021-05-19T02:57:05Z 2021-05-19T02:57:05Z CONTRIBUTOR  

JSON Test File (test.json):

json [ { "id": 123, "text": "FR Théâtre" }, { "id": 223, "text": [ "FR Théâtre" ] } ]

Command to import:

bash sqlite-utils insert test.db text test.json --pk=id

Resulting table view from datasette:

Original, db.py line 2225:

python return json.dumps(value, default=repr)

Fix, db.py line 2225:

python return json.dumps(value, default=repr, ensure_ascii=False)

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/257/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
807433181 MDU6SXNzdWU4MDc0MzMxODE= 1224 can't start immutable databases from configuration dir mode camallen 295329 closed 0     0 2021-02-12T17:50:13Z 2021-03-29T00:17:31Z 2021-03-29T00:17:31Z CONTRIBUTOR  

Say I have a /databases/ directory with multiple sqlite db files in that dir (1.db & 2.db) and an inspect-data.json file.

If I start datasette via datasette -h 0.0.0.0 /databases/ then the resulting databases are set to is_mutable: true as inspected via http://127.0.0.1:8001/-/databases.json

I don't want to have to list out the databases by name, e.g. datasette -i /databases/1.db -i /databases/2.db as i want the system to autodetect the sqlite dbs i have in the configuration directory

According to the docs outlined in https://docs.datasette.io/en/latest/settings.html?highlight=immutable#configuration-directory-mode this should be possible

inspect-data.json the result of running datasette inspect - any database files listed here will be treated as immutable, so they should not be changed while Datasette is running

I believe that if the inspect-json.json file present, then in theory the databases will be automatically set to immutable via this code https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L211-L216

However it appears the Click Multiple Options will return a tuple via https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/cli.py#L311-L317

The resulting tuple is passed to the Datasette app via kwargs and overrides the behaviour to set the databases to immutable via this arg https://github.com/simonw/datasette/blob/9603d893b9b72653895318c9104d754229fdb146/datasette/app.py#L182

If you think this is a bug and needs fixing, I am willing to make a PR to check for the empty immutable tuple before calling the Datasette class initializer as I think leaving that class interface alone is the best path here.

Thoughts?

Also - i'm loving Datasette, it truly is a wonderful tool, thank you :)

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1224/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
763207948 MDU6SXNzdWU3NjMyMDc5NDg= 1141 Default styling for bullet point lists simonw 9599 closed 0     0 2020-12-12T02:49:33Z 2021-03-29T00:14:05Z 2021-03-29T00:14:05Z OWNER  

I just noticed that https://datasette.io/content/recent_releases (which uses datasette-render-markdown) is missing its bullet points:

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1141/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
808036774 MDU6SXNzdWU4MDgwMzY3NzQ= 232 Run tests against Windows in GitHub Actions simonw 9599 closed 0     0 2021-02-14T20:09:45Z 2021-02-14T20:39:55Z 2021-02-14T20:39:55Z OWNER  

I'm going to try and get the test suite to run in Windows on GitHub Actions.

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/225#issuecomment-778834504

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/232/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
802583450 MDU6SXNzdWU4MDI1ODM0NTA= 226 3.4 release is broken - includes a rogue line simonw 9599 closed 0     0 2021-02-06T02:08:01Z 2021-02-06T02:10:26Z 2021-02-06T02:10:26Z OWNER  

I started seeing weird errors, caused by this line: https://github.com/simonw/sqlite-utils/blob/f8010ca78fed8c5fca6cde19658ec09fdd468420/sqlite_utils/cli.py#L1-L3

That was added by accident in 1b666f9315d4ea6bb332b2e75e48480c26100199

I'm surprised the tests didn't catch this!

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/226/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
783714076 MDU6SXNzdWU3ODM3MTQwNzY= 1184 request.full_path property simonw 9599 closed 0   Datasette 0.54 6346396 0 2021-01-11T21:21:58Z 2021-01-24T21:21:16Z 2021-01-11T21:34:47Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1184/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
760605882 MDU6SXNzdWU3NjA2MDU4ODI= 1135 Feature: --create option to create database file if it does not yet exist simonw 9599 closed 0     0 2020-12-09T19:23:58Z 2021-01-24T21:19:39Z 2020-12-09T19:45:52Z OWNER  

I'd like to be able to tell people to run the following in the Datasette documentation to get started:

brew install datasette
datasette install datasette-upload-csvs
datasette data.db --create --root --open

This would give them a local Datasette instance with the ability to drag-and-drop CSV files directly into it.

Just one catch: I don't want to have to talk them through creating an empty SQLite database file. So I want to add a new --create option which means "If any of the database files passed on the command-line do not yet exist, create them".

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1135/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
787900412 MDU6SXNzdWU3ODc5MDA0MTI= 222 .m2m() should accept alter=True parameter simonw 9599 closed 0     0 2021-01-18T04:15:43Z 2021-01-18T04:26:10Z 2021-01-18T04:26:10Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/222/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
783910901 MDU6SXNzdWU3ODM5MTA5MDE= 221 .add_missing_columns() does not take case insensitivity into account simonw 9599 closed 0     0 2021-01-12T05:01:00Z 2021-01-12T23:17:33Z 2021-01-12T23:17:33Z OWNER  

SQLite columns are case insensitive - but the .add_missing_columns() method doesn't know that. This means that it can crash if it identifies a column that is a case-insensitive duplicate of an existing column. https://github.com/simonw/sqlite-utils/blob/4cc82fd0bccc9d2eeb3510beb4e691d7da099f84/sqlite_utils/db.py#L1974-L1980

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/221/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
777530107 MDU6SXNzdWU3Nzc1MzAxMDc= 214 sqlite-utils enable-counts command simonw 9599 closed 0     0 2021-01-02T21:45:48Z 2021-01-03T04:26:44Z 2021-01-03T04:26:44Z OWNER  

The CLI version of #212 and #213.

# Enable counts for all tables:
sqlite-utils enable-counts data.db

# Enable counts for specific tables:
sqlite-utils enable-counts data.db table1 table2
sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/214/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
777386465 MDU6SXNzdWU3NzczODY0NjU= 211 table.triggers_dict introspection property simonw 9599 closed 0     0 2021-01-02T02:04:00Z 2021-01-02T02:10:10Z 2021-01-02T02:10:10Z OWNER  

table.triggers currently returns a list of Trigger values. A table.triggers_dict property could behave like columns_dict, returning a dictionary mapping trigger names to their SQL definitions for that table.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/211/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
769282206 MDU6SXNzdWU3NjkyODIyMDY= 30 Upgrade to sqlite-utils 3.0 (tests are failing) simonw 9599 closed 0     0 2020-12-16T21:25:15Z 2020-12-16T21:27:11Z 2020-12-16T21:27:10Z MEMBER  

``` results = beta_db["search_index"].search("run") if use_porter: assert results == [ ( "dogs.db/dogs", "1", "Cleo", "2020-08-22 04:41:33", 1, 0, "running", None, None, ) ] else:

      assert results == []

E assert <generator ob...x7f3c080a3e08> == [] E +<generator object Table.search at 0x7f3c080a3e08> E -[] E Full diff: E - [] E + <generator object Table.search at 0x7f3c080a3e08> ``` This was caused by a backwards incompatible change in sqlite-utils 3.0: https://sqlite-utils.readthedocs.io/en/stable/changelog.html#v3-0

 dogsheep-beta 197431109 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/30/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
758944006 MDU6SXNzdWU3NTg5NDQwMDY= 57 --readme throws 404 error if README does not exist in repo simonw 9599 closed 0     0 2020-12-07T23:58:49Z 2020-12-16T18:17:54Z 2020-12-16T18:17:54Z MEMBER  

It should fail silently (populate the column with a null) instead.

 github-to-sqlite 207052882 issue     
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/57/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
761713079 MDU6SXNzdWU3NjE3MTMwNzk= 1138 "Powered by Datasette" should link to new datasette.io site simonw 9599 closed 0     0 2020-12-10T23:33:41Z 2020-12-15T02:28:10Z 2020-12-10T23:37:14Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1138/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
761706858 MDU6SXNzdWU3NjE3MDY4NTg= 1137 Update README to reflect new datasette.io site simonw 9599 closed 0     0 2020-12-10T23:22:06Z 2020-12-10T23:28:50Z 2020-12-10T23:28:50Z OWNER  

Can finally close #659.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1137/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
753788261 MDU6SXNzdWU3NTM3ODgyNjE= 1118 messagge_is_html typo simonw 9599 closed 0     0 2020-11-30T20:43:22Z 2020-11-30T21:24:28Z 2020-11-30T21:24:28Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1118/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
753122082 MDU6SXNzdWU3NTMxMjIwODI= 56 Link to example tables from the README simonw 9599 closed 0     0 2020-11-30T04:01:51Z 2020-11-30T04:10:27Z 2020-11-30T04:10:27Z MEMBER  

Would help demonstrate how the tool works.

 github-to-sqlite 207052882 issue     
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/56/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
753026388 MDU6SXNzdWU3NTMwMjYzODg= 55 github-to-sqlite workflows does not correctly replace existing records simonw 9599 closed 0     0 2020-11-29T21:58:43Z 2020-11-29T23:48:50Z 2020-11-29T23:48:50Z MEMBER   github-to-sqlite 207052882 issue     
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/55/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
735644513 MDU6SXNzdWU3MzU2NDQ1MTM= 1081 Fixtures should use FTS4 or FTS5, not FTS3 simonw 9599 closed 0   Datasette 0.52 6055094 0 2020-11-03T21:24:13Z 2020-11-12T00:03:00Z 2020-11-12T00:02:59Z OWNER  

Just spotted that fixtures.db uses FTS3, which is pretty much obsolete these days.

https://github.com/simonw/datasette/blob/13d1228d80c91d382a05b1a9549ed02c300ef851/tests/fixtures.py#L488-L489

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1081/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
735648209 MDU6SXNzdWU3MzU2NDgyMDk= 193 --tsv output format option simonw 9599 closed 0   3.0 6079500 0 2020-11-03T21:31:18Z 2020-11-07T00:09:52Z 2020-11-07T00:09:52Z OWNER  

We already support --csv for output, and the insert command accepts --tsv. The output format options should accept --tsv too.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/193/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
733805089 MDU6SXNzdWU3MzM4MDUwODk= 1076 Release notes for 0.51 simonw 9599 closed 0   0.51 6026070 0 2020-10-31T20:51:21Z 2020-10-31T22:27:00Z 2020-10-31T22:27:00Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1076/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
724264574 MDU6SXNzdWU3MjQyNjQ1NzQ= 52 Option to fetch README and/or HTML-rendered README for repos simonw 9599 closed 0     0 2020-10-19T05:10:24Z 2020-10-19T05:33:42Z 2020-10-19T05:33:42Z MEMBER  

I'm thinking:

github-to-sqlite repos ... --readme # Populates readme column with raw text
github-to-sqlite repos ... --readme-html # Populates readme_html column with raw HTML

https://developer.github.com/v3/repos/contents/#get-a-repository-readme

 github-to-sqlite 207052882 issue     
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/52/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
718952107 MDU6SXNzdWU3MTg5NTIxMDc= 185 Use db[table] consistently in documentation simonw 9599 closed 0     0 2020-10-11T23:39:12Z 2020-10-12T00:13:41Z 2020-10-12T00:13:41Z OWNER  

The Python docs have a bunch of examples like this: https://sqlite-utils.readthedocs.io/en/stable/python-api.html python dogs.enable_fts(["name", "twitter"], create_triggers=True) This would be easier for people to understand if it looked like this instead: python db["dogs"].enable_fts(["name", "twitter"], create_triggers=True)

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/185/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
718938321 MDU6SXNzdWU3MTg5MzgzMjE= 3 Use a content hash for the note IDs simonw 9599 closed 0     0 2020-10-11T22:13:46Z 2020-10-11T23:15:04Z 2020-10-11T23:15:04Z MEMBER  

Without a GUID note IDs are pretty ineffective, but using a hash of the contents will at least avoid creating identical duplicates in the future.

https://sqlite-utils.readthedocs.io/en/stable/python-api.html#setting-an-id-based-on-the-hash-of-the-row-contents

 evernote-to-sqlite 303218369 issue     
{
    "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/3/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
718938046 MDU6SXNzdWU3MTg5MzgwNDY= 2 Convert dates to a better format simonw 9599 closed 0     0 2020-10-11T22:12:33Z 2020-10-11T23:15:03Z 2020-10-11T23:15:03Z MEMBER   evernote-to-sqlite 303218369 issue     
{
    "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/2/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
718255803 MDU6SXNzdWU3MTgyNTU4MDM= 1004 Replace MockRequest with Request.fake() simonw 9599 closed 0   Datasette 0.50 5971510 0 2020-10-09T15:55:28Z 2020-10-09T16:26:24Z 2020-10-09T16:26:24Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1004/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
713304417 MDU6SXNzdWU3MTMzMDQ0MTc= 989 Column action sort descending/ascending links should remove _next= pagination simonw 9599 closed 0   Datasette 0.50 5971510 0 2020-10-02T02:33:48Z 2020-10-08T23:55:15Z 2020-10-04T18:05:28Z OWNER  

On page https://latest.datasette.io/fixtures/sortable?_next=15%2Cg%2Cz&_sort=sortable clicking on sortable_with_nulls > sort_ascending links to https://latest.datasette.io/fixtures/sortable?_next=15%2Cg%2Cz&_sort_desc=sortable_with_nulls - which doesn't make sense.

Changing the sort order needs to reset to the first page.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/989/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
708185405 MDU6SXNzdWU3MDgxODU0MDU= 975 Dependabot couldn't authenticate with https://pypi.python.org/simple/ dependabot-preview[bot] 27856297 closed 0     0 2020-09-24T13:44:40Z 2020-09-25T13:34:34Z 2020-09-25T13:34:34Z CONTRIBUTOR  

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/975/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
706091046 MDU6SXNzdWU3MDYwOTEwNDY= 165 Make .transform() a keyword arguments only function simonw 9599 closed 0   2.20 5897911 0 2020-09-22T05:37:29Z 2020-09-24T20:35:47Z 2020-09-22T06:39:12Z OWNER  

And rename the first argument from columns= to types=

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/165/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
706757891 MDU6SXNzdWU3MDY3NTc4OTE= 169 Progress bar for "sqlite-utils extract" simonw 9599 closed 0   2.20 5897911 0 2020-09-22T23:40:21Z 2020-09-24T20:34:50Z 2020-09-23T00:02:40Z OWNER  

Since these operations could take a long time against large tables, it would be neat if there was a progress bar option for the CLI command.

The operations are full table scans so calculating progress shouldn't be too difficult. Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/42#issuecomment-513246831

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/169/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
703970713 MDU6SXNzdWU3MDM5NzA3MTM= 23 Sort option should persist between multiple searches simonw 9599 closed 0     0 2020-09-17T23:21:26Z 2020-09-18T22:39:12Z 2020-09-18T22:39:12Z MEMBER  

Following #21

dogsheep-beta 197431109 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/23/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
703962917 MDU6SXNzdWU3MDM5NjI5MTc= 22 Bug: UI says sorted by relevance in timeline view simonw 9599 closed 0     0 2020-09-17T23:02:07Z 2020-09-17T23:13:14Z 2020-09-17T23:13:14Z MEMBER  

In regular timeline view sort defaults to newest, not relevance - so this UI is incorrect:

 dogsheep-beta 197431109 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/22/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
703951918 MDU6SXNzdWU3MDM5NTE5MTg= 21 Option to sort search results by date simonw 9599 closed 0     0 2020-09-17T22:32:39Z 2020-09-17T22:55:35Z 2020-09-17T22:55:35Z MEMBER  

Sometimes I want to sort by date, not by relevance.

 dogsheep-beta 197431109 issue     
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/21/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
701584448 MDU6SXNzdWU3MDE1ODQ0NDg= 966 Remove _request_ip example from canned queries documentation simonw 9599 closed 0     0 2020-09-15T03:51:33Z 2020-09-15T03:52:45Z 2020-09-15T03:52:45Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/966/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
691475400 MDU6SXNzdWU2OTE0NzU0MDA= 958 Upgrade to latest Black (20.8b1) simonw 9599 closed 0   Datasette 0.49 5818042 0 2020-09-02T22:24:19Z 2020-09-11T21:34:24Z 2020-09-02T22:25:10Z OWNER  

Black has some changes: https://black.readthedocs.io/en/stable/change_log.html#b0 - in particular:

  • re-implemented support for explicit trailing commas: now it works consistently within any bracket pair, including nested structures (#1288 and duplicates)
  • Black now reindents docstrings when reindenting code around it (#1053)
 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/958/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
695359607 MDU6SXNzdWU2OTUzNTk2MDc= 150 Feature for tracing SQL queries simonw 9599 closed 0     0 2020-09-07T19:43:08Z 2020-09-07T21:57:01Z 2020-09-07T21:57:01Z OWNER  

Debugging sqlite-utils when something weird happens (e.g. #149) can be a bit tricky since it runs a bunch of different SQL statements behind the scenes.

An optional "tracing" mechanism for seeing what SQL is being executed would be useful.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/150/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);