home / github

Menu
  • Search all tables
  • GraphQL API

issues

Table actions
  • GraphQL API for issues

494 rows where comments = 1 and user = 9599 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: author_association, draft, created_at (date), updated_at (date), closed_at (date)

repo 12

  • datasette 354
  • sqlite-utils 96
  • twitter-to-sqlite 11
  • dogsheep-beta 8
  • github-to-sqlite 8
  • dogsheep-photos 7
  • healthkit-to-sqlite 4
  • swarm-to-sqlite 2
  • google-takeout-to-sqlite 1
  • pocket-to-sqlite 1
  • hacker-news-to-sqlite 1
  • evernote-to-sqlite 1

type 2

  • issue 448
  • pull 46

state 2

  • closed 387
  • open 107
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1903932086 PR_kwDOBm6k_c5aumyn 2192 Stop using parallel SQL queries for tables simonw 9599 closed 0     1 2023-09-20T01:28:43Z 2023-09-20T22:10:56Z 2023-09-20T22:10:55Z OWNER simonw/datasette/pulls/2192

Refs: - #2189


:books: Documentation preview :books:: https://datasette--2192.org.readthedocs.build/en/2192/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2192/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1886783150 PR_kwDOCGYnMM5Z1H1d 593 .transform() now preserves rowid values, refs #592 simonw 9599 closed 0     1 2023-09-08T01:02:28Z 2023-09-10T17:44:59Z 2023-09-09T00:45:30Z OWNER simonw/sqlite-utils/pulls/593

Refs: - #592

  • [x] Tests against weird shaped tables

I need to test that this works against:

  • rowid tables
  • Tables that have a column called rowid even though they are not rowid tables

:books: Documentation preview :books:: https://sqlite-utils--593.org.readthedocs.build/en/593/

sqlite-utils 140912432 pull    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/593/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1874255116 I_kwDOBm6k_c5vtt0M 2164 Ability to only load a specific list of plugins simonw 9599 closed 0     1 2023-08-30T19:33:41Z 2023-09-08T04:35:46Z 2023-08-30T22:12:27Z OWNER  

I'm going to try and get this working through an environment variable, so that you can start Datasette and it will only load a subset of plugins including those that use the register_commands() hook.

Initial research on this: - https://github.com/pytest-dev/pluggy/issues/422

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2164/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1884408624 I_kwDOBm6k_c5wUcsw 2177 Move schema tables from _internal to _catalog simonw 9599 open 0     1 2023-09-06T16:58:33Z 2023-09-06T17:04:30Z   OWNER  

This came up in discussion over: - https://github.com/simonw/datasette/pull/2174

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2177/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1879214365 I_kwDOCGYnMM5wAokd 590 Ability to tell if a Database is an in-memory one simonw 9599 open 0     1 2023-09-03T19:50:15Z 2023-09-03T19:50:36Z   OWNER  

Currently the constructor accepts memory=True or memory_name=... and uses those to create a connection, but does not record what those values were:

https://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L307-L349

This makes it hard to tell if a database object is to an in-memory or a file-based database, which is sometimes useful to know.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/590/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1292370469 I_kwDOBm6k_c5NCAIl 1765 Document plugins providing new plugin hook- simonw 9599 closed 0     1 2022-07-03T17:05:14Z 2023-08-31T23:08:24Z 2023-08-31T23:06:31Z OWNER  

I've used this pattern twice now: https://til.simonwillison.net/datasette/register-new-plugin-hooks - in datasette-graphql and datasette-low-disk-space-hook. I should describe the pattern on https://docs.datasette.io/en/stable/writing_plugins.html

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1765/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
742041667 MDU6SXNzdWU3NDIwNDE2Njc= 1092 Make cascading permission checks available to plugins simonw 9599 closed 0     1 2020-11-13T01:02:55Z 2023-08-30T22:17:42Z 2023-08-30T22:17:41Z OWNER  

The BaseView class has a method for cascading permission checks, but it's not easily accessible to plugins.

https://github.com/simonw/datasette/blob/5eb8e9bf250b26e30b017d39a392c33973997656/datasette/views/base.py#L75-L99

This leaves plugins like datasette-graphql having to implement their own versions of this logic, which is bad: https://github.com/simonw/datasette-graphql/issues/65

First check view-database - if that says False then disallow access, if it says True then allow access. If it says None check view-instance.

This should become a supported API that plugins are encouraged to use.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1092/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1872043170 I_kwDOBm6k_c5vlRyi 2163 Rename core_X to catalog_X in the internals simonw 9599 closed 0     1 2023-08-29T16:45:00Z 2023-08-29T17:01:31Z 2023-08-29T17:01:31Z OWNER  

Discussed with Alex this morning. We think the American spelling is fine here (it's shorter than catalogue) and that it's a slightly less lazy name than core_.

Follows: - https://github.com/simonw/datasette/issues/2157

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2163/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1870345352 PR_kwDOBm6k_c5Y90K9 2161 -s/--setting x y gets merged into datasette.yml, refs #2143, #2156 simonw 9599 closed 0     1 2023-08-28T19:30:42Z 2023-08-28T20:06:15Z 2023-08-28T20:06:14Z OWNER simonw/datasette/pulls/2161

This change updates the -s/--setting option to datasette serve to allow it to be used to set arbitrarily complex nested settings in a way that is compatible with the new -c datasette.yml work happening in: - #2143

It will enable things like this: datasette data.db --setting plugins.datasette-ripgrep.path "/home/simon/code" For the moment though it just affects settings - so you can do this: datasette data.db --setting settings.sql_time_limit_ms 3500 I've also implemented a backwards compatibility mechanism, so if you use it this way (the old way): datasette data.db --setting sql_time_limit_ms 3500 It will notice that the setting you passed is one of Datasette's core settings, and will treat that as if you said settings.sql_time_limit_ms instead.


:books: Documentation preview :books:: https://datasette--2161.org.readthedocs.build/en/2161/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2161/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1868713944 I_kwDOCGYnMM5vYk_Y 588 `table.get(column=value)` option for retrieving things not by their primary key simonw 9599 open 0     1 2023-08-28T00:41:23Z 2023-08-28T00:41:54Z   OWNER  

This came up working on this feature: - https://github.com/simonw/llm/pull/186

I have a table with this schema: sql CREATE TABLE [collections] ( [id] INTEGER PRIMARY KEY, [name] TEXT, [model] TEXT ); CREATE UNIQUE INDEX [idx_collections_name] ON [collections] ([name]); So the primary key is an integer (because it's going to have a huge number of rows foreign key related to it, and I don't want to store a larger text value thousands of times), but there is a unique constraint on the name - that would be the primary key column if not for all of those foreign keys.

Problem is, fetching the collection by name is actually pretty inconvenient.

Fetch by numeric ID:

python try: table["collections"].get(1) except NotFoundError: # It doesn't exist Fetching by name: python def get_collection(db, collection): rows = db["collections"].rows_where("name = ?", [collection]) try: return next(rows) except StopIteration: raise NotFoundError("Collection not found: {}".format(collection)) It would be neat if, for columns where we know that we should always get 0 or one result, we could do this instead: python try: collection = table["collections"].get(name="entries") except NotFoundError: # It doesn't exist The existing .get() method doesn't have any non-positional arguments, so using **kwargs like that should work:

https://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L1495

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/588/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
459689615 MDExOlB1bGxSZXF1ZXN0MjkwOTcxMjk1 524 Sort commits using isort, refs #516 simonw 9599 closed 0     1 2019-06-24T05:04:48Z 2023-08-23T01:31:08Z 2023-08-23T01:31:08Z OWNER simonw/datasette/pulls/524

Also added a lint unit test to ensure they stay sorted. #516

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/524/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1855836914 I_kwDOCGYnMM5undLy 583 Get rid of test.utils.collapse_whitespace simonw 9599 closed 0     1 2023-08-17T23:31:09Z 2023-08-18T00:59:19Z 2023-08-18T00:59:19Z OWNER  

I have a neater pattern for this now - instead of: https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_create.py#L472-L475

I now prefer:

https://github.com/simonw/sqlite-utils/blob/1dc6b5aa644a92d3654f7068110ed7930989ce71/tests/test_create.py#L1163-L1171

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/583/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1847201263 I_kwDOBm6k_c5uGg3v 2140 Remove all remaining documentation instances of '$ ' simonw 9599 closed 0     1 2023-08-11T17:42:13Z 2023-08-11T17:52:25Z 2023-08-11T17:45:00Z OWNER  

For example this: https://github.com/simonw/datasette/blob/4535568f2ce907af646304d0ebce2500ebd55677/docs/authentication.rst?plain=1#L33-L35

The problem with that $ prefix is that it prevents users from copying and pasting the raw command.

https://docs.datasette.io/en/stable/authentication.html#using-the-root-actor

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2140/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1843821954 I_kwDOBm6k_c5t5n2C 2137 Redesign row default JSON simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2023-08-09T18:49:11Z 2023-08-09T19:02:47Z   OWNER  

This URL here:

https://latest.datasette.io/fixtures/simple_primary_key/1.json?_extras=foreign_key_tables

json { "database": "fixtures", "table": "simple_primary_key", "rows": [ { "id": "1", "content": "hello" } ], "columns": [ "id", "content" ], "primary_keys": [ "id" ], "primary_key_values": [ "1" ], "units": {}, "foreign_key_tables": [ { "other_table": "foreign_key_references", "column": "id", "other_column": "foreign_key_with_blank_label", "count": 0, "link": "/fixtures/foreign_key_references?foreign_key_with_blank_label=1" }, { "other_table": "foreign_key_references", "column": "id", "other_column": "foreign_key_with_label", "count": 1, "link": "/fixtures/foreign_key_references?foreign_key_with_label=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f3", "count": 1, "link": "/fixtures/complex_foreign_keys?f3=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f2", "count": 0, "link": "/fixtures/complex_foreign_keys?f2=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f1", "count": 1, "link": "/fixtures/complex_foreign_keys?f1=1" } ], "query_ms": 4.226590999678592, "source": "tests/fixtures.py", "source_url": "https://github.com/simonw/datasette/blob/main/tests/fixtures.py", "license": "Apache License 2.0", "license_url": "https://github.com/simonw/datasette/blob/main/LICENSE", "ok": true, "truncated": false }

That ?_extras= should be ?_extra= - plus the row JSON should be redesigned to fit the new default JSON representation.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2137/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1560662739 I_kwDOBm6k_c5dBdLT 2007 `render_cell()` hook should take an optional `request` argument simonw 9599 closed 0     1 2023-01-28T03:13:00Z 2023-08-09T17:15:03Z 2023-01-28T03:34:26Z OWNER  

From Discord: https://discordapp.com/channels/823971286308356157/996877076982415491/1068227071156965486

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2007/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1840417903 I_kwDOBm6k_c5tsoxv 2131 Refactor code that supports templates_considered comment simonw 9599 open 0   Datasette 1.0 3268330 1 2023-08-08T01:28:36Z 2023-08-09T15:27:41Z   OWNER  

I ended up duplicating it here: https://github.com/simonw/datasette/blob/7532feb424b1dce614351e21b2265c04f9669fe2/datasette/views/database.py#L164-L167

I think it should move to datasette.render_template() - and maybe have a renamed template variable too.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2131/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1841343173 I_kwDOBm6k_c5twKrF 2132 Get form fields on query page working again simonw 9599 closed 0   Datasette 1.0a3 9700784 1 2023-08-08T13:39:05Z 2023-08-08T13:45:10Z 2023-08-08T13:45:09Z OWNER  

Caused by: - #2112

https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+where+%22pk1%22+%3D+%3Ap0+order+by+pk1%2C+pk2%2C+pk3+limit+101&p0=b

The :p0 form field is missing. Submitting the form results in this error:

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2132/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1840324765 I_kwDOBm6k_c5tsSCd 2129 CSV ?sql= should indicate errors simonw 9599 open 0   Datasette 1.0 3268330 1 2023-08-07T23:13:04Z 2023-08-08T02:02:21Z   OWNER  

https://latest.datasette.io/_memory.csv?sql=select+blah is a blank page right now:

bash curl -I 'https://latest.datasette.io/_memory.csv?sql=select+blah' HTTP/2 200 access-control-allow-origin: * access-control-allow-headers: Authorization, Content-Type access-control-expose-headers: Link access-control-allow-methods: GET, POST, HEAD, OPTIONS access-control-max-age: 3600 content-type: text/plain; charset=utf-8 x-databases: _memory, _internal, fixtures, fixtures2, extra_database, ephemeral date: Mon, 07 Aug 2023 23:12:15 GMT server: Google Frontend

Originally posted by @simonw in https://github.com/simonw/datasette/issues/2118#issuecomment-1668688947

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2129/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1822982933 I_kwDOBm6k_c5sqIMV 2117 Figure out what to do about `DatabaseView.name` simonw 9599 closed 0   Datasette 1.0a3 9700784 1 2023-07-26T18:58:06Z 2023-08-08T02:02:07Z 2023-08-08T02:02:07Z OWNER  

In the old code:

https://github.com/simonw/datasette/blob/08181823990a71ffa5a1b57b37259198eaa43e06/datasette/views/database.py#L34-L35

This name class attribute was later used by some of the plugin hooks, passed as view_name: https://github.com/simonw/datasette/blob/18dd88ee4d78fe9d760e9da96028ae06d938a85c/datasette/hookspecs.py#L50-L54

Figure out how that should work once I've refactored those classes to view functions instead.

Refs: - #2109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2117/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1823160748 I_kwDOCGYnMM5sqzms 581 `sqlite-utils convert --pdb` option simonw 9599 closed 0     1 2023-07-26T21:02:50Z 2023-07-26T21:07:45Z 2023-07-26T21:06:10Z OWNER  

While using sqlite-utils convert I realized it would be handy if you could pass --pdb to have it open the debugger at the first instance of a failed conversion.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/581/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1822936521 I_kwDOBm6k_c5sp83J 2110 Merge database index page and query view simonw 9599 closed 0   Datasette 1.0a3 9700784 1 2023-07-26T18:21:57Z 2023-07-26T19:53:25Z 2023-07-26T19:53:25Z OWNER  

Refs: - #2109

The idea here is that hitting /content without a ?sql= will show an empty result set AND default to including a bunch of extras about the list of tables in the database.

Then I won't have to think about /content and /content?sql= as separate pages any more.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2110/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1816876211 I_kwDOCGYnMM5sS1Sz 571 `.transform(keep_table=...)` option simonw 9599 closed 0     1 2023-07-22T19:49:29Z 2023-07-22T22:32:18Z 2023-07-22T22:32:18Z OWNER  

Also need a design for an option for the .transform() method to indicate that the new table should be created with a new name without dropping the old one.

I think keep_table="name_of_table" is good for this.

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/565#issuecomment-1646657324

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/571/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1205687423 I_kwDOCGYnMM5H3VR_ 426 CLI docs should link to Python docs and vice versa simonw 9599 closed 0 simonw 9599   1 2022-04-15T16:05:15Z 2023-07-22T22:13:22Z 2023-07-22T22:13:22Z OWNER  

For every command/API method there should be a link to the equivalent in the other form factor.

Maybe also link to the API and CLI reference pages too.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/426/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1803264272 I_kwDOBm6k_c5re6EQ 2101 alter: true support for JSON write API simonw 9599 open 0     1 2023-07-13T15:24:11Z 2023-07-13T15:24:18Z   OWNER  

Requested here: https://discord.com/channels/823971286308356157/823971286941302908/1129034187073134642

The former datasette-insert plugin had an option ?alter=1 to auto-add new columns. Does the JSON write API also have this?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2101/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1785360409 I_kwDOCGYnMM5qanAZ 563 `--empty-null` option when importing CSV simonw 9599 closed 0     1 2023-07-03T05:23:36Z 2023-07-03T05:44:43Z 2023-07-03T05:42:30Z OWNER  

CSV files with empty cells in (which come through as the empty string) are common and a bit gross.

Having an option that means "and if it's an empty string store null instead) would be cool.

I brainstormed name options here https://chat.openai.com/share/c947b738-ee7d-419c-af90-bc84e90987da

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/563/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1784794489 I_kwDOCGYnMM5qYc15 562 Explore the intersection between sqlite-utils and dataclasses simonw 9599 open 0     1 2023-07-02T19:23:08Z 2023-07-02T19:26:39Z   OWNER  

Aside: this makes me think it might be cool if sqlite-utils had a way of working with dataclasses rather than just dicts, and knew how to create a SQLite table to match a dataclass and maybe how to code-generate dataclasses for a specific table schema (dynamically or even using code-generation that can be written to disk, for better editor integrations).

Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1616742529

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/562/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1777548699 I_kwDOCGYnMM5p8z2b 561 `--stop-after` option for `insert` and `upsert` commands simonw 9599 closed 0     1 2023-06-27T18:44:15Z 2023-06-27T18:50:09Z 2023-06-27T18:50:08Z OWNER  

I found myself wanting to insert rows from a 849MB CSV file without processing the whole thing: https://huggingface.co/datasets/jerpint-org/HackAPrompt-Playground-Submissions/tree/main

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/561/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1718612569 I_kwDOCGYnMM5mb_JZ 552 Document how to setup shell auto-completion simonw 9599 closed 0     1 2023-05-21T19:20:41Z 2023-05-21T21:05:16Z 2023-05-21T21:03:40Z OWNER  

https://click.palletsprojects.com/en/8.1.x/shell-completion/

This works for zsh:

eval "$(_SQLITE_UTILS_COMPLETE=zsh_source sqlite-utils)"

This will probably work for bash:

eval "$(_SQLITE_UTILS_COMPLETE=bash_source sqlite-utils)"

Need to add this to the installation docs here: https://sqlite-utils.datasette.io/en/stable/installation.html - along with the pattern for adding that to .zshrc or whatever.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/552/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1718576761 I_kwDOCGYnMM5mb2Z5 548 analyze-tables should validate provide --column names simonw 9599 closed 0     1 2023-05-21T17:20:24Z 2023-05-21T17:35:52Z 2023-05-21T17:35:52Z OWNER  

Noticed this while testing: - #547

If you pass a non-existent column to -c/--column you don't get an error message.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/548/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1718572201 I_kwDOCGYnMM5mb1Sp 547 No need to show common values if everything is null simonw 9599 closed 0     1 2023-05-21T17:05:07Z 2023-05-21T17:19:21Z 2023-05-21T17:19:21Z OWNER  

Noticed this:

``` % sqlite-utils analyze-tables content.db repos -c delete_branch_on_merge --common-limit 20 --no-least repos.delete_branch_on_merge: (1/1)

Total rows: 158 Null rows: 158 Blank rows: 0

Distinct values: 0

Most common: 158: None ```

The 158: None there is duplicate information considering we already know there are 158/158 null rows.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/547/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1700936245 I_kwDOCGYnMM5lYjo1 542 Remove `skip_false=True` and `--no-skip-false` in `sqlite-utils` 4.0 simonw 9599 open 0   4.0 backwards incomatible changes 9374594 1 2023-05-08T21:04:28Z 2023-05-08T21:07:41Z   OWNER  

Following: - #527

The only reason I didn't remove fix this mis-feature entirely is that it represents a backwards incompatible change. I'll make that change in 4.0.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/542/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1700840265 I_kwDOCGYnMM5lYMNJ 541 Get tests to pass with `pytest -Werror` simonw 9599 open 0     1 2023-05-08T19:57:23Z 2023-05-08T19:59:35Z   OWNER  

Inspired by: - #534

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/541/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1649791661 I_kwDOBm6k_c5iVdKt 2050 Row page JSON should use new ?_extra= format simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2023-03-31T17:56:53Z 2023-03-31T17:59:49Z   OWNER  

https://latest.datasette.io/fixtures/facetable/2.json

Related: - #2049 - #1709

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2050/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1579973223 I_kwDOBm6k_c5eLHpn 2024 Mention WAL mode in documentation simonw 9599 open 0     1 2023-02-10T16:11:10Z 2023-02-10T16:11:53Z   OWNER  

It's not currently obvious from the docs how you can ensure that Datasette runs well in situations where other processes may update the underlying SQLite files.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2024/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
473288428 MDExOlB1bGxSZXF1ZXN0MzAxNDgzNjEz 564 First proof-of-concept of Datasette Library simonw 9599 open 0     1 2019-07-26T10:22:26Z 2023-02-07T15:14:11Z   OWNER simonw/datasette/pulls/564

Refs #417. Run it like this:

 datasette -d ~/Library

Uses a new plugin hook - available_databases()

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/564/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
1564769997 I_kwDOBm6k_c5dRH7N 2011 Applied facet did not result in an "x" icon to dismiss it simonw 9599 open 0     1 2023-01-31T17:57:44Z 2023-01-31T17:58:54Z   OWNER  

That's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata

It's for Contract Type of Non-Purchasing Contract (Rents, etc.) - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2011/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1563264257 I_kwDOBm6k_c5dLYUB 2010 Row page should default to card view simonw 9599 open 0   Datasette 1.0 3268330 1 2023-01-30T21:49:37Z 2023-01-30T21:52:06Z   OWNER  

Datasette currently uses the same table layout on the row pages as it does on the table pages:

https://datasette.io/content/pypi_packages?_sort=name&name__exact=datasette-column-inspect

https://datasette.io/content/pypi_packages/datasette-column-inspect

If you shrink down to mobile width you get this instead, on both of those pages:

I think that view, which I think of as the "card view", is plain better if you're looking at just a single row - and it (or a variant of it) should be the default presentation on the row page.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2010/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1557507274 I_kwDOBm6k_c5c1azK 2005 `extra_template_vars` should be OK to return `None` simonw 9599 open 0     1 2023-01-26T01:40:45Z 2023-01-26T01:41:50Z   OWNER  

Got this exception and had to make sure it always returned {}:

File ".../python3.11/site-packages/datasette/app.py", line 1049, in render_template assert isinstance(extra_vars, dict), "extra_vars is of type {}".format( AssertionError: extra_vars is of type <class 'NoneType'>

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2005/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1528448642 I_kwDOBm6k_c5bGkaC 1985 Don't let Datasette(path) without a list cause weird errors simonw 9599 closed 0     1 2023-01-11T05:17:44Z 2023-01-11T18:25:04Z 2023-01-11T18:25:04Z OWNER  

I got a confusing sqlite3.OperationalError: disk I/O error error in my tests, it turned out it was because this: python ds = Datasette(path) Should have been this: python ds = Datasette([path])

Originally posted by @simonw in https://github.com/simonw/datasette-faiss/issues/1#issuecomment-1378252673

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1985/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1526635374 PR_kwDOBm6k_c5HCCY2 1984 Upgrade Sphinx simonw 9599 closed 0     1 2023-01-10T02:00:40Z 2023-01-10T02:02:33Z 2023-01-10T02:02:33Z OWNER simonw/datasette/pulls/1984

Refs #1971


:books: Documentation preview :books:: https://datasette--1984.org.readthedocs.build/en/1984/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1984/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1426080014 I_kwDOBm6k_c5VAEEO 1867 /db/table/-/rename API (also allows atomic replace) simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-10-27T18:13:23Z 2023-01-09T15:34:12Z   OWNER  

There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first.

...

If people care about that kind of thing they could always push all of their inserts to a table called _tablename and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism).

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1867/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1524983536 I_kwDOBm6k_c5a5Wbw 1981 Canned query field labels truncated simonw 9599 open 0     1 2023-01-09T06:04:24Z 2023-01-09T06:05:44Z   OWNER  

Eg here on mobile: https://timezones.datasette.io/timezones/by_point?longitude=-0.1406632&latitude=50.8246776

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1981/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1115435536 I_kwDOBm6k_c5CfDIQ 1614 Try again with SQLite codemirror support simonw 9599 open 0     1 2022-01-26T20:05:20Z 2022-12-23T21:27:10Z   OWNER  

I tried and failed to implement autocomplete a while ago. Relevant code:

https://github.com/codemirror/legacy-modes/blob/8f36abca5f55024258cd23d9cfb0203d8d244f0d/mode/sql.js#L335

Sounds like upgrading to CodeMirror 6 ASAP would be worthwhile since it has better accessibility and touch screen support: https://codemirror.net/6/

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1614/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1501778647 I_kwDOBm6k_c5Zg1LX 1964 Cog menu is not keyboard accessible (also no ARIA) simonw 9599 open 0     1 2022-12-18T06:36:28Z 2022-12-18T06:37:28Z   OWNER  

This menu here: https://latest.datasette.io/fixtures/attraction_characteristic

You can tab to it (see the outline) and hit space or enter to open it, but you can't then navigate the items in the open menu using the keyboard.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1964/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1500636982 I_kwDOBm6k_c5Zcec2 1962 Alternative, async-friendly pattern for `make_app_client()` and similar - fully retire `TestClient` simonw 9599 open 0     1 2022-12-16T17:56:51Z 2022-12-16T21:55:29Z   OWNER  

In this issue I replaced a whole bunch of places that used the non-async app_client fixture with an async ds_client fixture instead: - #1959

But I didn't get everything, and a lot of tests are still using the old TestClient mechanism as a result.

The main work here is replacing all of the app_client_... fixtures which use variants on the default client - and changing the tests that call make_app_client() to do something else instead.

This requires some careful thought. I need to come up with a really nice pattern for creating variants on the ds_client default fixture - and do so in a way that minimizes the number of open files, refs:

  • 1843

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1962/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1490576818 I_kwDOBm6k_c5Y2GWy 1943 `/-/permissions` should list available permissions simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-12-11T23:38:03Z 2022-12-15T00:41:37Z   OWNER  

Idea: a /-/permissions introspection endpoint for listing registered permissions

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1939#issuecomment-1345691103

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1943/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1497577017 I_kwDOBm6k_c5ZQzY5 1957 Reconsider row value truncation on query page simonw 9599 open 0     1 2022-12-14T23:49:47Z 2022-12-14T23:50:50Z   OWNER  

Consider this example: https://ripgrep.datasette.io/repos?sql=select+json_group_array%28full_name%29+from+repos

sql select json_group_array(full_name) from repos

My intention here was to get a string of JSON I can copy and paste elsewhere - see: https://til.simonwillison.net/sqlite/compare-before-after-json

The truncation isn't helping here.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1957/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1495716243 I_kwDOBm6k_c5ZJtGT 1952 Improvements to /-/create-token restrictions interface simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-12-14T05:22:39Z 2022-12-14T05:23:13Z   OWNER  

It would be neat not to show write permissions against immutable databases too - and not hard from a performance perspective since it doesn't involve hundreds more permission checks.

That will need permissions to grow a flag for if they need a mutable database though, which is a bigger job.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1947#issuecomment-1350414402

Also, DO show the _memory database there if Datasette was started in --crossdb mode.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1952/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1468689139 I_kwDOBm6k_c5Ximrz 1914 Finalize design of JSON for Datasette 1.0 simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-11-29T20:59:10Z 2022-12-13T06:15:54Z   OWNER  

Tracking issue.

  • [ ] #1709
  • [ ] #1729
  • [ ] #1875
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1914/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1200649502 I_kwDOBm6k_c5HkHUe 1709 Redesigned JSON API with ?_extra= parameters simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-04-11T22:57:49Z 2022-12-13T05:29:06Z   OWNER  

This will be the single biggest breaking change for the 1.0 release.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1709/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1200650491 I_kwDOBm6k_c5HkHj7 1711 Template context powered entirely by the JSON API format simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-04-11T22:59:27Z 2022-12-13T05:29:06Z   OWNER  

Datasette 1.0 will have a stable template context. I'm going to achieve this by refactoring the templates to work only with keys returned by the API (or some of its extras) - then the API documentation will double up as template documentation.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1711/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1197926598 I_kwDOBm6k_c5HZujG 1705 How to upgrade your plugin for 1.0 documentation simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-04-08T23:16:47Z 2022-12-13T05:29:05Z   OWNER  

Among other things, needed by: - #1704

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1705/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1493404423 I_kwDOBm6k_c5ZA4sH 1948 500 error on permission debug page when testing actors with _r simonw 9599 open 0     1 2022-12-13T05:22:03Z 2022-12-13T05:22:19Z   OWNER  

The 500 error is silent unless you are looking at the DevTools network pane.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1948/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1487757143 I_kwDOCGYnMM5YrV9X 517 Drop support for Python 3.6 simonw 9599 closed 0     1 2022-12-10T01:23:31Z 2022-12-10T01:36:36Z 2022-12-10T01:36:36Z OWNER  

CI has started failing for Python 3.6: https://github.com/simonw/sqlite-utils/actions/runs/3576322798

It's fixable by swiching away from ubuntu-latest according to:

  • https://github.com/actions/setup-python/issues/355#issuecomment-1335042510

But https://endoflife.date/python says that 3.6 end of life was almost 6 years ago, and end of security support nearly 1 year ago.

So I'm OK dropping support entirely - Python 3.6 users will still be able to install version 3.30, just not any releases that come next.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/517/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1487738738 I_kwDOBm6k_c5YrRdy 1942 Option for plugins to request that JSON be served on the page simonw 9599 open 0   Datasette 1.0 3268330 1 2022-12-10T01:08:53Z 2022-12-10T01:11:30Z   OWNER  

Idea came from a conversation with @hydrosquall - what if a Datasette plugin could say "I'd like the JSON for a page to be included in a variable on the HTML page"?

datasette-cluster-map already needs this - the first thing it does when the page loads is fetch() a JSON representation of that same data.

This idea fits with my overall goals to unify the JSON and HTML context too.

Refs: - #1711

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1942/reactions",
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1486036269 I_kwDOBm6k_c5Ykx0t 1941 Mechanism for supporting key rotation for DATASETTE_SECRET simonw 9599 open 0     1 2022-12-09T05:24:53Z 2022-12-09T05:25:20Z   OWNER  

Currently if you change DATASETTE_SECRET all existing signed tokens - both cookies and API tokens and potentially other things too - will instantly expire.

Adding support for key rotation would allow keys to be rotated on a semi-regular basis without logging everyone out / invalidating every API token instantly.

Can model this on how Django does it: https://github.com/django/django/commit/0dcd549bbe36c060f536ec270d34d9e7d4b8e6c7

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1941/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1471969984 I_kwDOBm6k_c5XvHrA 1926 Release notes for 1.0a1 (and release it) simonw 9599 closed 0   Datasette 1.0a1 7867486 1 2022-12-01T21:18:12Z 2022-12-01T22:06:13Z 2022-12-01T22:06:12Z OWNER  

Mainly CORS support and a few small bug fixes.

Changes: https://github.com/simonw/datasette/compare/1.0a0...99da46f7258225fc6fd8e94ddc20859ccccc4109

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1926/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1469044738 I_kwDOBm6k_c5Xj9gC 1918 API explorer should list mutable databases first simonw 9599 closed 0   Datasette 1.0a1 7867486 1 2022-11-30T04:53:33Z 2022-11-30T05:22:07Z 2022-11-30T05:07:56Z OWNER  

https://latest.datasette.io/-/api hides ephemeral down at the bottom, would be more interesting if it was at the top.

Related: - #1915

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1918/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1469062686 I_kwDOBm6k_c5XkB4e 1919 Intermittent `test_delete_row` test failure simonw 9599 open 0     1 2022-11-30T05:18:46Z 2022-11-30T05:20:56Z   OWNER  

https://github.com/simonw/datasette/actions/runs/3580503393/jobs/6022689591

``` delete_response = await ds_write.client.post( "/data/{}/{}/-/delete".format(table, delete_path), headers={ "Authorization": "***".format(write_token(ds_write)), }, )

  assert delete_response.status_code == 200

E assert 404 == 200 E + where 404 = <Response [404 Not Found]>.status_code

/home/runner/work/datasette/datasette/tests/test_api_write.py:396: AssertionError =========================== short test summary info ============================ FAILED tests/test_api_write.py::test_delete_row[compound_pk_table-row_for_create2-pks2-article,k] - assert 404 == 200 + where 404 = <Response [404 Not Found]>.status_code ``` This passes most of the time, but very occasionally fails - in this case in Python 3.7

It seems to only fail for the article,k compound primary key test.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1919/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1469015001 I_kwDOBm6k_c5Xj2PZ 1916 GET requests against POST endpoints should not 500 error simonw 9599 closed 0   Datasette 1.0a1 7867486 1 2022-11-30T04:04:43Z 2022-11-30T05:15:19Z 2022-11-30T05:15:19Z OWNER  

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1916/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1432012302 I_kwDOBm6k_c5VWsYO 1877 Refactor and tidy up final write API code simonw 9599 closed 0     1 2022-11-01T20:00:11Z 2022-11-29T19:44:16Z 2022-11-29T19:44:07Z OWNER  
  • views/table.py has got a bit too big - I think the write classes should be pulled out into a separate module.
  • [x] There's duplicate logic for deciding if the table and database exist and checking permissions
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1877/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1468592292 PR_kwDOBm6k_c5D6nzE 1912 Merge 1.0-dev (with initial write API) back into main simonw 9599 closed 0     1 2022-11-29T19:31:21Z 2022-11-29T19:39:37Z 2022-11-29T19:39:36Z OWNER simonw/datasette/pulls/1912

See: - #1892


:books: Documentation preview :books:: https://datasette--1912.org.readthedocs.build/en/1912/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1912/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1466952626 I_kwDOBm6k_c5Xb-uy 1909 Option to sort facets alphabetically simonw 9599 open 0     1 2022-11-28T19:18:14Z 2022-11-28T19:19:26Z   OWNER  

Suggested here: - https://github.com/simonw/datasette/discussions/1908

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1909/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1447439985 I_kwDOBm6k_c5WRi5x 1888 API explorer should take immutability into account simonw 9599 closed 0   Datasette 1.0a0 8658075 1 2022-11-14T06:00:14Z 2022-11-15T19:59:10Z 2022-11-14T06:04:48Z OWNER  

Refs: - #1871

I noticed the API explorer doesn't show any links on https://latest-1-0-dev.datasette.io/-/api because the fixtures database is immutable.

It should still show read examples there.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1888/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1423347412 I_kwDOBm6k_c5U1o7U 1857 Prevent API tokens from using /-/create-token to create more tokens simonw 9599 closed 0   Datasette 1.0a0 8658075 1 2022-10-26T02:38:09Z 2022-11-15T19:57:11Z 2022-10-26T02:57:26Z OWNER  

It strikes me that users should NOT be able to use a token to create additional tokens.

The current design actually does allow that, since the dstok_ Bearer token can be used to authenticate calls to the /-/create-token page.

So I think I need a mechanism whereby that page can only allow access to users authenticated by cookie.

Not obvious how to do that though, since Datasette's authentication actor system is designed to abstract that detail away!

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1850#issuecomment-1291417100

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1857/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1425682079 I_kwDOBm6k_c5U-i6f 1865 Stop syncing main to master simonw 9599 closed 0     1 2022-10-27T13:55:38Z 2022-10-27T13:58:27Z 2022-10-27T13:56:13Z OWNER  

I think it's been long enough now that I can drop the code that syncs the main branch to master.

I originally added this for people who might be using datasette publish ... --branch master - which might only have been me anyway!

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1865/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
642297505 MDU6SXNzdWU2NDIyOTc1MDU= 857 Comprehensive documentation for variables made available to templates simonw 9599 closed 0   Datasette 1.0 3268330 1 2020-06-20T03:19:43Z 2022-10-26T02:58:17Z 2022-10-26T02:58:17Z OWNER  

Needed for the Datasette 1.0 release, so template authors can trust that Datasette is unlikely to break their templates.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/857/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1423069384 I_kwDOCGYnMM5U0lDI 504 db.close() method, calling db.conn.close() simonw 9599 closed 0     1 2022-10-25T20:50:50Z 2022-10-25T21:00:29Z 2022-10-25T20:57:47Z OWNER  

I ended up needing to use db.conn.close() to fix this issue: - #503

I think .close() should be a method on Database itself.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/504/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1422954582 I_kwDOCGYnMM5U0JBW 502 Fix tests for Python 3.11 simonw 9599 closed 0     1 2022-10-25T19:20:31Z 2022-10-25T19:23:47Z 2022-10-25T19:23:47Z OWNER  

The way errors are represented has changed: https://github.com/simonw/sqlite-utils/actions/runs/3323588047/jobs/5494127154 ``` ____ testquery_invalid_function ______

db_path = '/tmp/pytest-of-runner/pytest-0/test_query_invalid_function0/test.db'

def test_query_invalid_function(db_path):
    result = CliRunner().invoke(
        cli.cli, [db_path, "select bad()", "--functions", "def invalid_python"]
    )
    assert result.exit_code == 1
  assert (
        result.output.strip()
        == "Error: Error in functions definition: invalid syntax (<string>, line 1)"
    )

E AssertionError: assert 'Error: Error...ing>, line 1)' == 'Error: Error...ing>, line 1)' E - Error: Error in functions definition: invalid syntax (<string>, line 1) E ? ^^^^^^ ^^^^^^ E + Error: Error in functions definition: expected '(' (<string>, line 1) E ? ^^^^^^^ ^^^ ```

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/502/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1386734383 I_kwDOBm6k_c5Sp-Mv 1821 Release Datasette 0.63a0 simonw 9599 closed 0     1 2022-09-26T21:15:27Z 2022-09-26T22:06:39Z 2022-09-26T22:06:39Z OWNER  
  • The prepare_jinja2_environment(env, datasette) plugin hook now accepts an optional datasette argument. Hook implementations can also now return an async function which will be awaited automatically. (#1809)
  • --load-extension option now supports entrypoints. Thanks, Alex Garcia. (#1789)
  • New tutorial: Cleaning data with sqlite-utils and Datasette.
  • Facet size can now be set per-table with the new facet_size table metadata option. (#1804)
  • truncate_cells_html setting now also affects long URLs in columns. (#1805)
  • Database(is_mutable=) now defaults to True. (#1808)
  • Non-JavaScript textarea now increases height to fit the SQL query. (#1786)
  • More detailed command descriptions on the CLI reference page. (#1787)
  • Datasette no longer enforces upper bounds on its depenedencies. (#1800)
  • Facets are now displayed with better line-breaks in long values. Thanks, Daniel Rech. (#1794)
  • The settings.json file used in Configuration directory mode is now validated on startup. (#1816)
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1821/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1386530156 I_kwDOCGYnMM5SpMVs 492 Idea: ability to pass extra variables to `--convert` scripts simonw 9599 open 0     1 2022-09-26T18:30:45Z 2022-09-26T18:33:19Z   OWNER  

Got this idea from this example in https://jeqo.github.io/notes/2022-09-24-ingest-logs-sqlite/

bash sqlite-utils insert /tmp/kafka-logs.db logs server.log.2022-09-24-21 --text --convert " import re r = re.compile(r'^\[(?P<datetime>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\] (?P<level>\w+) (?P<log>(.+(\n(?\!\[).+|)+))', re.MULTILINE) def convert(text): rows = [m.groupdict() for m in r.finditer(text)] for row in rows: row.update({'server': 'localhost'}) row.update({'component': 'broker'}) return rows " And the accompanying note:

The row.update allows to label rows as I’m planning to ingest logs from different hosts and potentially different components.

This made me think: it might be neat if you could inject additional variable values into that script with extra command-line options, to make this kind of reuse easier. Something like this:

bash sqlite-utils insert /tmp/kafka-logs.db logs server.log.2022-09-24-21 --text --convert " import re r = re.compile(r'^\[(?P<datetime>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\] (?P<level>\w+) (?P<log>(.+(\n(?\!\[).+|)+))', re.MULTILINE) def convert(text): rows = [m.groupdict() for m in r.finditer(text)] for row in rows: row.update({'server': server}) row.update({'component': component}) return rows " --var server "localhost" --var component "broker"

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/492/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1366915240 I_kwDOBm6k_c5ReXio 1807 Plugin ecosystem needs to avoid crashes due to no available databases simonw 9599 open 0     1 2022-09-08T19:54:34Z 2022-09-08T20:14:05Z   OWNER  

Opening this here to track the issue first reported in: - https://github.com/simonw/datasette-upload-dbs/issues/5

Plugins that expect to be able to write to a database need to not crash in situations where no writable database is available.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1807/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1362567197 PR_kwDOBm6k_c4-ZxWD 1803 Workaround for test failure: RuntimeError: There is no current event loop simonw 9599 closed 0     1 2022-09-06T00:31:06Z 2022-09-06T00:40:19Z 2022-09-06T00:40:19Z OWNER simonw/datasette/pulls/1803

Closes #1802


:books: Documentation preview :books:: https://datasette--1803.org.readthedocs.build/en/1803/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1803/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1362367821 PR_kwDOBm6k_c4-ZGW6 1801 Don't use upper bound dependencies, refs #1800 simonw 9599 closed 0     1 2022-09-05T18:29:28Z 2022-09-05T18:35:41Z 2022-09-05T18:35:41Z OWNER simonw/datasette/pulls/1801

See https://iscinumpy.dev/post/bound-version-constraints/


:books: Documentation preview :books:: https://datasette--1801.org.readthedocs.build/en/1801/

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1801/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1353481513 I_kwDOCGYnMM5QrH0p 478 `sqlite-utils tables data.db table1 table2` simonw 9599 open 0     1 2022-08-28T22:05:53Z 2022-08-28T22:22:35Z   OWNER  

The sqlite-utils tables command currently lists all tables.

If you have a huge table in there then running it with --counts can get expensive, because of the huge table.

Would be useful if it could accept an optional list of tables that it should execute against, as an alternative to the default of all of them.

This should be a backwards compatible change. Current design is: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#tables

``` Usage: sqlite-utils tables [OPTIONS] PATH

List the tables in the database

Example:

  sqlite-utils tables trees.db

```

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/478/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1353189941 I_kwDOCGYnMM5QqAo1 475 table.default_values introspection property simonw 9599 closed 0   3.29 8355157 1 2022-08-27T22:33:31Z 2022-08-27T22:44:46Z 2022-08-27T22:43:02Z OWNER  

Interesting challenge with default_value: I need to be able to tell if the default values passed to .create() differ from those in the database already.

Introspecting that is a bit tricky:

```pycon

import sqlite_utils db = sqlite_utils.Database(memory=True) db["blah"].create({"id": int, "name": str}, not_null=("name",), defaults={"name": "bob"}) <Table blah (id, name)> db["blah"].columns [Column(cid=0, name='id', type='INTEGER', notnull=0, default_value=None, is_pk=0), Column(cid=1, name='name', type='TEXT', notnull=1, default_value="'bob'", is_pk=0)] `` Note how a default value of the Python stringbobis represented in the results ofPRAGMA table_info()asdefault_value="'bob'"` - it's got single quotes added to it!

So comparing default values from introspecting the database needs me to first parse that syntax. This may require a new table introspection method.

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/468#issuecomment-1229279539

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/475/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1352953535 PR_kwDOCGYnMM4950Az 473 Support entrypoints for `--load-extension` simonw 9599 closed 0     1 2022-08-27T05:53:59Z 2022-08-27T05:55:52Z 2022-08-27T05:55:47Z OWNER simonw/sqlite-utils/pulls/473

Refs #470


:books: Documentation preview :books:: https://sqlite-utils--473.org.readthedocs.build/en/473/

sqlite-utils 140912432 pull    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/473/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
1352931464 I_kwDOCGYnMM5QpBiI 469 sqlite-utils rows --order option simonw 9599 closed 0   3.29 8355157 1 2022-08-27T03:49:51Z 2022-08-27T04:30:49Z 2022-08-27T04:10:32Z OWNER  

For consistency with search: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#search

-o, --order TEXT Order by ('column' or 'column desc')

I wanted to run sqlite-utils rows db.db mytable --order 'rowid desc' to see the most recently imported rows.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/469/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1320243134 I_kwDOCGYnMM5OsU-- 458 Support custom names for registered functions simonw 9599 closed 0   3.29 8355157 1 2022-07-28T00:13:00Z 2022-08-27T03:56:01Z 2022-07-28T00:13:57Z OWNER  

In this example:

```python @db.register_function def reverse_string(s): return "".join(reversed(list(s)))

print(db.execute('select reverse_string("hello")').fetchone()[0])

``` There's currently no way to over-ride the automatically selected name for the SQL function.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/458/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1338137350 I_kwDOBm6k_c5PwlsG 1781 Ensure Datasette Lite is promoted in docs and README simonw 9599 closed 0   Datasette 0.62 8303187 1 2022-08-14T05:12:35Z 2022-08-14T15:24:40Z 2022-08-14T15:24:40Z OWNER  

As of 0.62 https://lite.datasette.io is a supported piece of the overall Datasette ecosystem.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1781/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1326349129 I_kwDOCGYnMM5PDntJ 461 Consider including animated SVG console demos simonw 9599 open 0     1 2022-08-02T20:10:04Z 2022-08-02T20:12:14Z   OWNER  

I recorded this one using https://github.com/nbedos/termtosvg - with pipx install termtosvg and then termtosvg - execute demo - exit to save.

json [ { "id": 1, "name": "Catimus" }, { "id": 2, "name": "Feliopia" } ]

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/461/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
663145122 MDU6SXNzdWU2NjMxNDUxMjI= 903 Add temporary plugin testing pattern to the testing docs simonw 9599 closed 0     1 2020-07-21T16:22:34Z 2022-07-18T21:34:33Z 2022-07-18T21:31:22Z OWNER  

https://til.simonwillison.net/pytest/registering-plugins-in-tests

Would be useful to include this pattern on https://datasette.readthedocs.io/en/stable/testing_plugins.html

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/903/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1306548397 I_kwDOCGYnMM5N4Fit 454 CLI command for duplicating tables simonw 9599 closed 0     1 2022-07-15T21:31:27Z 2022-07-15T21:48:23Z 2022-07-15T21:45:51Z OWNER  

CLI equivalent of: - #449

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/454/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1243512344 I_kwDOBm6k_c5KHn4Y 1747 Add tutorials to the getting started guide simonw 9599 closed 0     1 2022-05-20T19:01:52Z 2022-05-20T19:12:30Z 2022-05-20T19:05:34Z OWNER  

On https://docs.datasette.io/en/stable/getting_started.html

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1747/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1239080102 I_kwDOBm6k_c5J2tym 1745 Documentation on running cog simonw 9599 closed 0     1 2022-05-17T19:41:06Z 2022-05-17T19:45:51Z 2022-05-17T19:43:45Z OWNER  

Noticed that cog -r docs/*.rst isn't documented in https://docs.datasette.io/en/latest/contributing.html#editing-and-building-the-documentation

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1745/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1223263540 I_kwDOBm6k_c5I6YU0 1735 Datasette setting to disable threading (for Pyodide) simonw 9599 closed 0     1 2022-05-02T19:31:08Z 2022-05-02T23:25:49Z 2022-05-02T20:13:52Z OWNER  

I'm going to add a Datasette setting to disable threading entirely, designed for usage in this particular case.

I thought about adding a new setting, then I noticed this:

datasette mydatabase.db --setting num_sql_threads 10

I'm going to let users set that to 0 to disable threaded execution of SQL queries.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1733#issuecomment-1115278325

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1735/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1215216249 I_kwDOCGYnMM5Ibrp5 428 Research adding support for savepoints simonw 9599 open 0     1 2022-04-26T01:04:01Z 2022-04-26T01:05:29Z   OWNER  

https://www.sqlite.org/lang_savepoint.html

Savepoints are like regular transactions except they have names and can be nested.

Would there be any value in adding support to them to sqlite-utils, potentially as some kind of context manager? Something like this: python with db.savepoint("name"): # do stuff with db.savepoint("name2"): # do more stuff raise Release # Rolls back to before "name2" savepoint I've never used this feature so I'm not comfortable adding anything like this without a bunch of extra research.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/428/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1212838949 I_kwDOBm6k_c5ISnQl 1716 Configure git blame to ignore Black commit simonw 9599 closed 0     1 2022-04-22T21:56:37Z 2022-04-22T22:02:19Z 2022-04-22T22:02:19Z OWNER  

GitHub can support this in blame views now too:

https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1716/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1200866134 I_kwDOCGYnMM5Hk8NW 424 Better error message if you try to create a table with no columns simonw 9599 closed 0     1 2022-04-12T02:43:20Z 2022-04-13T22:40:15Z 2022-04-13T22:40:10Z OWNER  

Seen here:

  • https://github.com/simonw/geojson-to-sqlite/issues/30

Attempting to create a table with no columns produced this confusing error:

File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/geojson_to_sqlite/utils.py", line 69, in import_features db[table].create(column_types, pk=pk) File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 863, in create self.db.create_table( File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 517, in create_table self.execute(sql) File "/Users/simon/.local/pipx/venvs/geojson-to-sqlite/lib/python3.9/site-packages/sqlite_utils/db.py", line 236, in execute return self.conn.execute(sql) sqlite3.OperationalError: near ")": syntax error

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/424/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1200649889 I_kwDOBm6k_c5HkHah 1710 Guide for plugin authors to upgrade their plugins for 1.0 simonw 9599 closed 0     1 2022-04-11T22:58:25Z 2022-04-11T23:04:01Z 2022-04-11T23:03:25Z OWNER  

I'll also encourage testing against both Datasette 0.x and Datasette 1.0 using a GitHub Actions matrix.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1710/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1190828163 I_kwDOBm6k_c5G-piD 1698 Add a warning about bots and Cloud Run simonw 9599 closed 0     1 2022-04-03T05:57:17Z 2022-04-03T06:10:24Z 2022-04-03T06:10:24Z OWNER  

Recommend the https://github.com/simonw/datasette-block-robots plugin if you are going to run a large database in Cloud Run (one with a lot of rows).

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1698/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1189113609 I_kwDOBm6k_c5G4G8J 1697 `Request.fake(..., url_vars={})` simonw 9599 closed 0   Datasette 1.0 3268330 1 2022-04-01T01:48:40Z 2022-04-01T02:02:18Z 2022-04-01T02:02:10Z OWNER  

I just created an alternative .fake() method because I wanted to fake the url_vars captured in the route as well: ```python from datasette.utils.asgi import Request class Request(Request):

@classmethod
def fake(cls, path_with_query_string, method="GET", scheme="http", url_vars=None):
    """Useful for constructing Request objects for tests"""
    path, _, query_string = path_with_query_string.partition("?")
    scope = {
        "http_version": "1.1",
        "method": method,
        "path": path,
        "raw_path": path_with_query_string.encode("latin-1"),
        "query_string": query_string.encode("latin-1"),
        "scheme": scheme,
        "type": "http",
    }
    if url_vars:
        scope["url_route"] = {
            "kwargs": url_vars
        }
    return cls(scope, None)

```

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1697/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1182065616 I_kwDOBm6k_c5GdOPQ 1689 datasette.add_message() documentation is incorrect simonw 9599 closed 0     1 2022-03-26T20:49:42Z 2022-03-26T21:35:57Z 2022-03-26T20:51:21Z OWNER  

https://docs.datasette.io/en/0.61.1/internals.html#add-message-request-message-message-type-datasette-info says:

.add_message(request, message, message_type=datasette.INFO)

But in the code it's:

https://github.com/simonw/datasette/blob/6b99e4a66ba0ed8fca8ee41ceb7206928b60d5d1/datasette/app.py#L582

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1689/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1181236173 I_kwDOCGYnMM5GaDvN 422 Reconsider not running convert functions against null values simonw 9599 open 0     1 2022-03-25T20:22:40Z 2022-03-25T20:23:21Z   OWNER  

I just got caught out by the fact that None values are not processed by the .convert() mechanism https://github.com/simonw/sqlite-utils/blob/0b7b80bd40fe86e4d66a04c9f607d94991c45c0b/sqlite_utils/db.py#L2504-L2510

I had run this code while working on #420 and I wasn't sure why it didn't work:

``` $ sqlite-utils add-column content.db articles score float $ sqlite-utils convert content.db articles score ' import random random.seed(10)

def convert(value): global random return random.random() ' `` The reason it didn't work is that the newly addedscorecolumn was full ofnull` values.

I fixed it by doing this instead:

$ sqlite-utils add-column content.db articles score float --not-null-default 1.0

But this indicates to me that the design of convert() here may be incorrect.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/422/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1179998071 I_kwDOBm6k_c5GVVd3 1684 Mechanism for disabling faceting on large tables only simonw 9599 open 0     1 2022-03-24T20:06:11Z 2022-03-24T20:13:19Z   OWNER  

Forest turned off faceting on https://labordata.bunkum.us/ because it was causing performance problems on some of the huge tables - but it would be nice if it could still be an option on smaller tables such as https://labordata.bunkum.us/voluntary_recognitions-4421085/voluntary_recognitions

One option: a new setting that automatically disables faceting (and facet suggestion) for tables that have either more than X rows or that are so big that the count could not be completed within the time limit.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1684/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1178521513 I_kwDOBm6k_c5GPs-p 1682 SQL queries against databases with different routes are broken simonw 9599 closed 0     1 2022-03-23T18:42:57Z 2022-03-23T18:48:16Z 2022-03-23T18:48:16Z OWNER  

500 error on https://datasette-hashed-urls-preview.vercel.app/fixtures-09f8f95?sql=select+*+from+facetable

Here's the trace: File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py", line 54, in data return await QueryView(self.ds).data( File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/views/database.py", line 232, in data self.ds.get_database(database), sql File "/Users/simon/.local/share/virtualenvs/datasette-hashed-urls-ssI2fO50/lib/python3.10/site-packages/datasette/app.py", line 401, in get_database return self.databases[name] KeyError: 'fixtures-aa7318b' It looks like this is a Datasette bug, which is frustrating because I just shipped Datasette 0.61 five minutes ago!

Originally posted by @simonw in https://github.com/simonw/datasette-hashed-urls/issues/13#issuecomment-1076693667

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1682/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1175715988 I_kwDOBm6k_c5GFACU 1678 Make `check_visibility()` a documented API simonw 9599 closed 0   Datasette 1.0 3268330 1 2022-03-21T17:30:34Z 2022-03-21T19:04:03Z 2022-03-21T19:01:46Z OWNER  

Spotted this while working on: - #1677

https://github.com/simonw/datasette/blob/e627510b760198ccedba9e5af47a771e847785c9/datasette/utils/init.py#L1005-L1021

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1678/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
1175694248 I_kwDOBm6k_c5GE6uo 1677 Remove `check_permission()` from `BaseView` simonw 9599 closed 0   Datasette 1.0 3268330 1 2022-03-21T17:18:18Z 2022-03-21T18:45:04Z 2022-03-21T18:45:03Z OWNER  

Follow-on from: - #1675

Refs: - #1660

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1677/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed
780153562 MDU6SXNzdWU3ODAxNTM1NjI= 1177 Ability to stream all rows as newline-delimited JSON simonw 9599 open 0   Datasette 1.0 3268330 1 2021-01-06T07:10:48Z 2022-03-21T15:08:52Z   OWNER  

Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas:

pandas.read_json("https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on", lines=True)

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1101#issuecomment-755128038

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1177/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1174717287 I_kwDOBm6k_c5GBMNn 1674 Tweak design of /.json simonw 9599 open 0   Datasette 1.0 3268330 1 2022-03-20T22:58:01Z 2022-03-20T22:58:40Z   OWNER  

https://latest.datasette.io/.json

Currently: json { "_memory": { "name": "_memory", "hash": null, "color": "a6c7b9", "path": "/_memory", "tables_and_views_truncated": [], "tables_and_views_more": false, "tables_count": 0, "table_rows_sum": 0, "show_table_row_counts": false, "hidden_table_rows_sum": 0, "hidden_tables_count": 0, "views_count": 0, "private": false }, "fixtures": { "name": "fixtures", "hash": "645005884646eb941c89997fbd1c0dd6be517cb1b493df9816ae497c0c5afbaa", "color": "645005", "path": "/fixtures", "tables_and_views_truncated": [ { "name": "compound_three_primary_keys", "columns": [ "pk1", "pk2", "pk3", "content" ], "primary_keys": [ "pk1", "pk2", "pk3" ], "count": 1001, "hidden": false, "fts_table": null, "num_relationships_for_sorting": 0, "private": false }, As-of this issue the "path" key is confusing, it doesn't match what https://latest.datasette.io/-/databases returns:

  • 1668

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1674/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);
Powered by Datasette · Queries took 292.408ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows