home / github

Menu
  • Search all tables
  • GraphQL API

issues

Table actions
  • GraphQL API for issues

107 rows where comments = 1, state = "open" and user = 9599 sorted by updated_at descending

✖
✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: milestone, author_association, draft, created_at (date), updated_at (date)

repo 9

  • datasette 84
  • sqlite-utils 12
  • dogsheep-photos 3
  • twitter-to-sqlite 2
  • github-to-sqlite 2
  • dogsheep-beta 1
  • google-takeout-to-sqlite 1
  • pocket-to-sqlite 1
  • evernote-to-sqlite 1

type 2

  • issue 104
  • pull 3

state 1

  • open · 107 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1884408624 I_kwDOBm6k_c5wUcsw 2177 Move schema tables from _internal to _catalog simonw 9599 open 0     1 2023-09-06T16:58:33Z 2023-09-06T17:04:30Z   OWNER  

This came up in discussion over: - https://github.com/simonw/datasette/pull/2174

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2177/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1879214365 I_kwDOCGYnMM5wAokd 590 Ability to tell if a Database is an in-memory one simonw 9599 open 0     1 2023-09-03T19:50:15Z 2023-09-03T19:50:36Z   OWNER  

Currently the constructor accepts memory=True or memory_name=... and uses those to create a connection, but does not record what those values were:

https://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L307-L349

This makes it hard to tell if a database object is to an in-memory or a file-based database, which is sometimes useful to know.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/590/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1868713944 I_kwDOCGYnMM5vYk_Y 588 `table.get(column=value)` option for retrieving things not by their primary key simonw 9599 open 0     1 2023-08-28T00:41:23Z 2023-08-28T00:41:54Z   OWNER  

This came up working on this feature: - https://github.com/simonw/llm/pull/186

I have a table with this schema: sql CREATE TABLE [collections] ( [id] INTEGER PRIMARY KEY, [name] TEXT, [model] TEXT ); CREATE UNIQUE INDEX [idx_collections_name] ON [collections] ([name]); So the primary key is an integer (because it's going to have a huge number of rows foreign key related to it, and I don't want to store a larger text value thousands of times), but there is a unique constraint on the name - that would be the primary key column if not for all of those foreign keys.

Problem is, fetching the collection by name is actually pretty inconvenient.

Fetch by numeric ID:

python try: table["collections"].get(1) except NotFoundError: # It doesn't exist Fetching by name: python def get_collection(db, collection): rows = db["collections"].rows_where("name = ?", [collection]) try: return next(rows) except StopIteration: raise NotFoundError("Collection not found: {}".format(collection)) It would be neat if, for columns where we know that we should always get 0 or one result, we could do this instead: python try: collection = table["collections"].get(name="entries") except NotFoundError: # It doesn't exist The existing .get() method doesn't have any non-positional arguments, so using **kwargs like that should work:

https://github.com/simonw/sqlite-utils/blob/1260bdc7bfe31c36c272572c6389125f8de6ef71/sqlite_utils/db.py#L1495

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/588/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1843821954 I_kwDOBm6k_c5t5n2C 2137 Redesign row default JSON simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2023-08-09T18:49:11Z 2023-08-09T19:02:47Z   OWNER  

This URL here:

https://latest.datasette.io/fixtures/simple_primary_key/1.json?_extras=foreign_key_tables

json { "database": "fixtures", "table": "simple_primary_key", "rows": [ { "id": "1", "content": "hello" } ], "columns": [ "id", "content" ], "primary_keys": [ "id" ], "primary_key_values": [ "1" ], "units": {}, "foreign_key_tables": [ { "other_table": "foreign_key_references", "column": "id", "other_column": "foreign_key_with_blank_label", "count": 0, "link": "/fixtures/foreign_key_references?foreign_key_with_blank_label=1" }, { "other_table": "foreign_key_references", "column": "id", "other_column": "foreign_key_with_label", "count": 1, "link": "/fixtures/foreign_key_references?foreign_key_with_label=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f3", "count": 1, "link": "/fixtures/complex_foreign_keys?f3=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f2", "count": 0, "link": "/fixtures/complex_foreign_keys?f2=1" }, { "other_table": "complex_foreign_keys", "column": "id", "other_column": "f1", "count": 1, "link": "/fixtures/complex_foreign_keys?f1=1" } ], "query_ms": 4.226590999678592, "source": "tests/fixtures.py", "source_url": "https://github.com/simonw/datasette/blob/main/tests/fixtures.py", "license": "Apache License 2.0", "license_url": "https://github.com/simonw/datasette/blob/main/LICENSE", "ok": true, "truncated": false }

That ?_extras= should be ?_extra= - plus the row JSON should be redesigned to fit the new default JSON representation.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2137/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1840417903 I_kwDOBm6k_c5tsoxv 2131 Refactor code that supports templates_considered comment simonw 9599 open 0   Datasette 1.0 3268330 1 2023-08-08T01:28:36Z 2023-08-09T15:27:41Z   OWNER  

I ended up duplicating it here: https://github.com/simonw/datasette/blob/7532feb424b1dce614351e21b2265c04f9669fe2/datasette/views/database.py#L164-L167

I think it should move to datasette.render_template() - and maybe have a renamed template variable too.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2131/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1840324765 I_kwDOBm6k_c5tsSCd 2129 CSV ?sql= should indicate errors simonw 9599 open 0   Datasette 1.0 3268330 1 2023-08-07T23:13:04Z 2023-08-08T02:02:21Z   OWNER  

https://latest.datasette.io/_memory.csv?sql=select+blah is a blank page right now:

bash curl -I 'https://latest.datasette.io/_memory.csv?sql=select+blah' HTTP/2 200 access-control-allow-origin: * access-control-allow-headers: Authorization, Content-Type access-control-expose-headers: Link access-control-allow-methods: GET, POST, HEAD, OPTIONS access-control-max-age: 3600 content-type: text/plain; charset=utf-8 x-databases: _memory, _internal, fixtures, fixtures2, extra_database, ephemeral date: Mon, 07 Aug 2023 23:12:15 GMT server: Google Frontend

Originally posted by @simonw in https://github.com/simonw/datasette/issues/2118#issuecomment-1668688947

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2129/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1803264272 I_kwDOBm6k_c5re6EQ 2101 alter: true support for JSON write API simonw 9599 open 0     1 2023-07-13T15:24:11Z 2023-07-13T15:24:18Z   OWNER  

Requested here: https://discord.com/channels/823971286308356157/823971286941302908/1129034187073134642

The former datasette-insert plugin had an option ?alter=1 to auto-add new columns. Does the JSON write API also have this?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2101/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1784794489 I_kwDOCGYnMM5qYc15 562 Explore the intersection between sqlite-utils and dataclasses simonw 9599 open 0     1 2023-07-02T19:23:08Z 2023-07-02T19:26:39Z   OWNER  

Aside: this makes me think it might be cool if sqlite-utils had a way of working with dataclasses rather than just dicts, and knew how to create a SQLite table to match a dataclass and maybe how to code-generate dataclasses for a specific table schema (dynamically or even using code-generation that can be written to disk, for better editor integrations).

Originally posted by @simonw in https://github.com/simonw/llm/issues/65#issuecomment-1616742529

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/562/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1700936245 I_kwDOCGYnMM5lYjo1 542 Remove `skip_false=True` and `--no-skip-false` in `sqlite-utils` 4.0 simonw 9599 open 0   4.0 backwards incomatible changes 9374594 1 2023-05-08T21:04:28Z 2023-05-08T21:07:41Z   OWNER  

Following: - #527

The only reason I didn't remove fix this mis-feature entirely is that it represents a backwards incompatible change. I'll make that change in 4.0.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/542/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1700840265 I_kwDOCGYnMM5lYMNJ 541 Get tests to pass with `pytest -Werror` simonw 9599 open 0     1 2023-05-08T19:57:23Z 2023-05-08T19:59:35Z   OWNER  

Inspired by: - #534

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/541/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1649791661 I_kwDOBm6k_c5iVdKt 2050 Row page JSON should use new ?_extra= format simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2023-03-31T17:56:53Z 2023-03-31T17:59:49Z   OWNER  

https://latest.datasette.io/fixtures/facetable/2.json

Related: - #2049 - #1709

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2050/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1579973223 I_kwDOBm6k_c5eLHpn 2024 Mention WAL mode in documentation simonw 9599 open 0     1 2023-02-10T16:11:10Z 2023-02-10T16:11:53Z   OWNER  

It's not currently obvious from the docs how you can ensure that Datasette runs well in situations where other processes may update the underlying SQLite files.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2024/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
473288428 MDExOlB1bGxSZXF1ZXN0MzAxNDgzNjEz 564 First proof-of-concept of Datasette Library simonw 9599 open 0     1 2019-07-26T10:22:26Z 2023-02-07T15:14:11Z   OWNER simonw/datasette/pulls/564

Refs #417. Run it like this:

 datasette -d ~/Library

Uses a new plugin hook - available_databases()

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/564/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
1  
1564769997 I_kwDOBm6k_c5dRH7N 2011 Applied facet did not result in an "x" icon to dismiss it simonw 9599 open 0     1 2023-01-31T17:57:44Z 2023-01-31T17:58:54Z   OWNER  

That's against this data https://data.sfgov.org/City-Management-and-Ethics/Supplier-Contracts/cqi5-hm2d imported using https://datasette.io/plugins/datasette-socrata

It's for Contract Type of Non-Purchasing Contract (Rents, etc.) - so possible that some of the spaces or punctuation in either the name of the value tripped up the code that decides if the X icon should be displayed.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2011/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1563264257 I_kwDOBm6k_c5dLYUB 2010 Row page should default to card view simonw 9599 open 0   Datasette 1.0 3268330 1 2023-01-30T21:49:37Z 2023-01-30T21:52:06Z   OWNER  

Datasette currently uses the same table layout on the row pages as it does on the table pages:

https://datasette.io/content/pypi_packages?_sort=name&name__exact=datasette-column-inspect

https://datasette.io/content/pypi_packages/datasette-column-inspect

If you shrink down to mobile width you get this instead, on both of those pages:

I think that view, which I think of as the "card view", is plain better if you're looking at just a single row - and it (or a variant of it) should be the default presentation on the row page.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2010/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1557507274 I_kwDOBm6k_c5c1azK 2005 `extra_template_vars` should be OK to return `None` simonw 9599 open 0     1 2023-01-26T01:40:45Z 2023-01-26T01:41:50Z   OWNER  

Got this exception and had to make sure it always returned {}:

File ".../python3.11/site-packages/datasette/app.py", line 1049, in render_template assert isinstance(extra_vars, dict), "extra_vars is of type {}".format( AssertionError: extra_vars is of type <class 'NoneType'>

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2005/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1426080014 I_kwDOBm6k_c5VAEEO 1867 /db/table/-/rename API (also allows atomic replace) simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-10-27T18:13:23Z 2023-01-09T15:34:12Z   OWNER  

There's one catch with batched inserts: if your CLI tool fails half way through you could end up with a partially populated table - since a bunch of batches will have succeeded first.

...

If people care about that kind of thing they could always push all of their inserts to a table called _tablename and then atomically rename that once they've uploaded all of the data (assuming I provide an atomic-rename-this-table mechanism).

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1866#issuecomment-1293893789

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1867/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1524983536 I_kwDOBm6k_c5a5Wbw 1981 Canned query field labels truncated simonw 9599 open 0     1 2023-01-09T06:04:24Z 2023-01-09T06:05:44Z   OWNER  

Eg here on mobile: https://timezones.datasette.io/timezones/by_point?longitude=-0.1406632&latitude=50.8246776

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1981/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1115435536 I_kwDOBm6k_c5CfDIQ 1614 Try again with SQLite codemirror support simonw 9599 open 0     1 2022-01-26T20:05:20Z 2022-12-23T21:27:10Z   OWNER  

I tried and failed to implement autocomplete a while ago. Relevant code:

https://github.com/codemirror/legacy-modes/blob/8f36abca5f55024258cd23d9cfb0203d8d244f0d/mode/sql.js#L335

Sounds like upgrading to CodeMirror 6 ASAP would be worthwhile since it has better accessibility and touch screen support: https://codemirror.net/6/

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1614/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1501778647 I_kwDOBm6k_c5Zg1LX 1964 Cog menu is not keyboard accessible (also no ARIA) simonw 9599 open 0     1 2022-12-18T06:36:28Z 2022-12-18T06:37:28Z   OWNER  

This menu here: https://latest.datasette.io/fixtures/attraction_characteristic

You can tab to it (see the outline) and hit space or enter to open it, but you can't then navigate the items in the open menu using the keyboard.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1964/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1500636982 I_kwDOBm6k_c5Zcec2 1962 Alternative, async-friendly pattern for `make_app_client()` and similar - fully retire `TestClient` simonw 9599 open 0     1 2022-12-16T17:56:51Z 2022-12-16T21:55:29Z   OWNER  

In this issue I replaced a whole bunch of places that used the non-async app_client fixture with an async ds_client fixture instead: - #1959

But I didn't get everything, and a lot of tests are still using the old TestClient mechanism as a result.

The main work here is replacing all of the app_client_... fixtures which use variants on the default client - and changing the tests that call make_app_client() to do something else instead.

This requires some careful thought. I need to come up with a really nice pattern for creating variants on the ds_client default fixture - and do so in a way that minimizes the number of open files, refs:

  • 1843

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1962/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1490576818 I_kwDOBm6k_c5Y2GWy 1943 `/-/permissions` should list available permissions simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-12-11T23:38:03Z 2022-12-15T00:41:37Z   OWNER  

Idea: a /-/permissions introspection endpoint for listing registered permissions

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1939#issuecomment-1345691103

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1943/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1497577017 I_kwDOBm6k_c5ZQzY5 1957 Reconsider row value truncation on query page simonw 9599 open 0     1 2022-12-14T23:49:47Z 2022-12-14T23:50:50Z   OWNER  

Consider this example: https://ripgrep.datasette.io/repos?sql=select+json_group_array%28full_name%29+from+repos

sql select json_group_array(full_name) from repos

My intention here was to get a string of JSON I can copy and paste elsewhere - see: https://til.simonwillison.net/sqlite/compare-before-after-json

The truncation isn't helping here.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1957/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1495716243 I_kwDOBm6k_c5ZJtGT 1952 Improvements to /-/create-token restrictions interface simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-12-14T05:22:39Z 2022-12-14T05:23:13Z   OWNER  

It would be neat not to show write permissions against immutable databases too - and not hard from a performance perspective since it doesn't involve hundreds more permission checks.

That will need permissions to grow a flag for if they need a mutable database though, which is a bigger job.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1947#issuecomment-1350414402

Also, DO show the _memory database there if Datasette was started in --crossdb mode.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1952/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1468689139 I_kwDOBm6k_c5Ximrz 1914 Finalize design of JSON for Datasette 1.0 simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-11-29T20:59:10Z 2022-12-13T06:15:54Z   OWNER  

Tracking issue.

  • [ ] #1709
  • [ ] #1729
  • [ ] #1875
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1914/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1200649502 I_kwDOBm6k_c5HkHUe 1709 Redesigned JSON API with ?_extra= parameters simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-04-11T22:57:49Z 2022-12-13T05:29:06Z   OWNER  

This will be the single biggest breaking change for the 1.0 release.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1709/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1200650491 I_kwDOBm6k_c5HkHj7 1711 Template context powered entirely by the JSON API format simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-04-11T22:59:27Z 2022-12-13T05:29:06Z   OWNER  

Datasette 1.0 will have a stable template context. I'm going to achieve this by refactoring the templates to work only with keys returned by the API (or some of its extras) - then the API documentation will double up as template documentation.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1711/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1197926598 I_kwDOBm6k_c5HZujG 1705 How to upgrade your plugin for 1.0 documentation simonw 9599 open 0   Datasette 1.0a-next 8755003 1 2022-04-08T23:16:47Z 2022-12-13T05:29:05Z   OWNER  

Among other things, needed by: - #1704

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1705/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1493404423 I_kwDOBm6k_c5ZA4sH 1948 500 error on permission debug page when testing actors with _r simonw 9599 open 0     1 2022-12-13T05:22:03Z 2022-12-13T05:22:19Z   OWNER  

The 500 error is silent unless you are looking at the DevTools network pane.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1948/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1487738738 I_kwDOBm6k_c5YrRdy 1942 Option for plugins to request that JSON be served on the page simonw 9599 open 0   Datasette 1.0 3268330 1 2022-12-10T01:08:53Z 2022-12-10T01:11:30Z   OWNER  

Idea came from a conversation with @hydrosquall - what if a Datasette plugin could say "I'd like the JSON for a page to be included in a variable on the HTML page"?

datasette-cluster-map already needs this - the first thing it does when the page loads is fetch() a JSON representation of that same data.

This idea fits with my overall goals to unify the JSON and HTML context too.

Refs: - #1711

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1942/reactions",
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1486036269 I_kwDOBm6k_c5Ykx0t 1941 Mechanism for supporting key rotation for DATASETTE_SECRET simonw 9599 open 0     1 2022-12-09T05:24:53Z 2022-12-09T05:25:20Z   OWNER  

Currently if you change DATASETTE_SECRET all existing signed tokens - both cookies and API tokens and potentially other things too - will instantly expire.

Adding support for key rotation would allow keys to be rotated on a semi-regular basis without logging everyone out / invalidating every API token instantly.

Can model this on how Django does it: https://github.com/django/django/commit/0dcd549bbe36c060f536ec270d34d9e7d4b8e6c7

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1941/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1469062686 I_kwDOBm6k_c5XkB4e 1919 Intermittent `test_delete_row` test failure simonw 9599 open 0     1 2022-11-30T05:18:46Z 2022-11-30T05:20:56Z   OWNER  

https://github.com/simonw/datasette/actions/runs/3580503393/jobs/6022689591

``` delete_response = await ds_write.client.post( "/data/{}/{}/-/delete".format(table, delete_path), headers={ "Authorization": "***".format(write_token(ds_write)), }, )

  assert delete_response.status_code == 200

E assert 404 == 200 E + where 404 = <Response [404 Not Found]>.status_code

/home/runner/work/datasette/datasette/tests/test_api_write.py:396: AssertionError =========================== short test summary info ============================ FAILED tests/test_api_write.py::test_delete_row[compound_pk_table-row_for_create2-pks2-article,k] - assert 404 == 200 + where 404 = <Response [404 Not Found]>.status_code ``` This passes most of the time, but very occasionally fails - in this case in Python 3.7

It seems to only fail for the article,k compound primary key test.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1919/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1466952626 I_kwDOBm6k_c5Xb-uy 1909 Option to sort facets alphabetically simonw 9599 open 0     1 2022-11-28T19:18:14Z 2022-11-28T19:19:26Z   OWNER  

Suggested here: - https://github.com/simonw/datasette/discussions/1908

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1909/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1386530156 I_kwDOCGYnMM5SpMVs 492 Idea: ability to pass extra variables to `--convert` scripts simonw 9599 open 0     1 2022-09-26T18:30:45Z 2022-09-26T18:33:19Z   OWNER  

Got this idea from this example in https://jeqo.github.io/notes/2022-09-24-ingest-logs-sqlite/

bash sqlite-utils insert /tmp/kafka-logs.db logs server.log.2022-09-24-21 --text --convert " import re r = re.compile(r'^\[(?P<datetime>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\] (?P<level>\w+) (?P<log>(.+(\n(?\!\[).+|)+))', re.MULTILINE) def convert(text): rows = [m.groupdict() for m in r.finditer(text)] for row in rows: row.update({'server': 'localhost'}) row.update({'component': 'broker'}) return rows " And the accompanying note:

The row.update allows to label rows as I’m planning to ingest logs from different hosts and potentially different components.

This made me think: it might be neat if you could inject additional variable values into that script with extra command-line options, to make this kind of reuse easier. Something like this:

bash sqlite-utils insert /tmp/kafka-logs.db logs server.log.2022-09-24-21 --text --convert " import re r = re.compile(r'^\[(?P<datetime>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3})\] (?P<level>\w+) (?P<log>(.+(\n(?\!\[).+|)+))', re.MULTILINE) def convert(text): rows = [m.groupdict() for m in r.finditer(text)] for row in rows: row.update({'server': server}) row.update({'component': component}) return rows " --var server "localhost" --var component "broker"

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/492/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1366915240 I_kwDOBm6k_c5ReXio 1807 Plugin ecosystem needs to avoid crashes due to no available databases simonw 9599 open 0     1 2022-09-08T19:54:34Z 2022-09-08T20:14:05Z   OWNER  

Opening this here to track the issue first reported in: - https://github.com/simonw/datasette-upload-dbs/issues/5

Plugins that expect to be able to write to a database need to not crash in situations where no writable database is available.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1807/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1353481513 I_kwDOCGYnMM5QrH0p 478 `sqlite-utils tables data.db table1 table2` simonw 9599 open 0     1 2022-08-28T22:05:53Z 2022-08-28T22:22:35Z   OWNER  

The sqlite-utils tables command currently lists all tables.

If you have a huge table in there then running it with --counts can get expensive, because of the huge table.

Would be useful if it could accept an optional list of tables that it should execute against, as an alternative to the default of all of them.

This should be a backwards compatible change. Current design is: https://sqlite-utils.datasette.io/en/stable/cli-reference.html#tables

``` Usage: sqlite-utils tables [OPTIONS] PATH

List the tables in the database

Example:

  sqlite-utils tables trees.db

```

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/478/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1326349129 I_kwDOCGYnMM5PDntJ 461 Consider including animated SVG console demos simonw 9599 open 0     1 2022-08-02T20:10:04Z 2022-08-02T20:12:14Z   OWNER  

I recorded this one using https://github.com/nbedos/termtosvg - with pipx install termtosvg and then termtosvg - execute demo - exit to save.

json [ { "id": 1, "name": "Catimus" }, { "id": 2, "name": "Feliopia" } ]

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/461/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1215216249 I_kwDOCGYnMM5Ibrp5 428 Research adding support for savepoints simonw 9599 open 0     1 2022-04-26T01:04:01Z 2022-04-26T01:05:29Z   OWNER  

https://www.sqlite.org/lang_savepoint.html

Savepoints are like regular transactions except they have names and can be nested.

Would there be any value in adding support to them to sqlite-utils, potentially as some kind of context manager? Something like this: python with db.savepoint("name"): # do stuff with db.savepoint("name2"): # do more stuff raise Release # Rolls back to before "name2" savepoint I've never used this feature so I'm not comfortable adding anything like this without a bunch of extra research.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/428/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1181236173 I_kwDOCGYnMM5GaDvN 422 Reconsider not running convert functions against null values simonw 9599 open 0     1 2022-03-25T20:22:40Z 2022-03-25T20:23:21Z   OWNER  

I just got caught out by the fact that None values are not processed by the .convert() mechanism https://github.com/simonw/sqlite-utils/blob/0b7b80bd40fe86e4d66a04c9f607d94991c45c0b/sqlite_utils/db.py#L2504-L2510

I had run this code while working on #420 and I wasn't sure why it didn't work:

``` $ sqlite-utils add-column content.db articles score float $ sqlite-utils convert content.db articles score ' import random random.seed(10)

def convert(value): global random return random.random() ' `` The reason it didn't work is that the newly addedscorecolumn was full ofnull` values.

I fixed it by doing this instead:

$ sqlite-utils add-column content.db articles score float --not-null-default 1.0

But this indicates to me that the design of convert() here may be incorrect.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/422/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1179998071 I_kwDOBm6k_c5GVVd3 1684 Mechanism for disabling faceting on large tables only simonw 9599 open 0     1 2022-03-24T20:06:11Z 2022-03-24T20:13:19Z   OWNER  

Forest turned off faceting on https://labordata.bunkum.us/ because it was causing performance problems on some of the huge tables - but it would be nice if it could still be an option on smaller tables such as https://labordata.bunkum.us/voluntary_recognitions-4421085/voluntary_recognitions

One option: a new setting that automatically disables faceting (and facet suggestion) for tables that have either more than X rows or that are so big that the count could not be completed within the time limit.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1684/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
780153562 MDU6SXNzdWU3ODAxNTM1NjI= 1177 Ability to stream all rows as newline-delimited JSON simonw 9599 open 0   Datasette 1.0 3268330 1 2021-01-06T07:10:48Z 2022-03-21T15:08:52Z   OWNER  

Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas:

pandas.read_json("https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on", lines=True)

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1101#issuecomment-755128038

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1177/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1174717287 I_kwDOBm6k_c5GBMNn 1674 Tweak design of /.json simonw 9599 open 0   Datasette 1.0 3268330 1 2022-03-20T22:58:01Z 2022-03-20T22:58:40Z   OWNER  

https://latest.datasette.io/.json

Currently: json { "_memory": { "name": "_memory", "hash": null, "color": "a6c7b9", "path": "/_memory", "tables_and_views_truncated": [], "tables_and_views_more": false, "tables_count": 0, "table_rows_sum": 0, "show_table_row_counts": false, "hidden_table_rows_sum": 0, "hidden_tables_count": 0, "views_count": 0, "private": false }, "fixtures": { "name": "fixtures", "hash": "645005884646eb941c89997fbd1c0dd6be517cb1b493df9816ae497c0c5afbaa", "color": "645005", "path": "/fixtures", "tables_and_views_truncated": [ { "name": "compound_three_primary_keys", "columns": [ "pk1", "pk2", "pk3", "content" ], "primary_keys": [ "pk1", "pk2", "pk3" ], "count": 1001, "hidden": false, "fts_table": null, "num_relationships_for_sorting": 0, "private": false }, As-of this issue the "path" key is confusing, it doesn't match what https://latest.datasette.io/-/databases returns:

  • 1668

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1674/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1174708375 I_kwDOBm6k_c5GBKCX 1673 Streaming CSV spends a lot of time in `table_column_details` simonw 9599 open 0     1 2022-03-20T22:25:28Z 2022-03-20T22:34:06Z   OWNER  

At least I think it does. I tried running py-spy top -p $PID against a Datasette process that was trying to do:

datasette covid.db --get '/covid/ny_times_us_counties.csv?_size=10&_stream=on'

While investigating: - #1355

And spotted this: ``` datasette covid.db --get /covid/ny_times_us_counties.csv?_size=10&_stream=on' (python v3.10.2) Total Samples 5800 GIL: 71.00%, Active: 98.00%, Threads: 4

%Own %Total OwnTime TotalTime Function (filename:line)
8.00% 8.00% 4.32s 4.38s sql_operation_in_thread (datasette/database.py:212) 5.00% 5.00% 3.77s 3.93s table_column_details (datasette/utils/init.py:614) 6.00% 6.00% 3.72s 3.72s _worker (concurrent/futures/thread.py:81) 7.00% 7.00% 2.98s 2.98s _read_from_self (asyncio/selector_events.py:120) 5.00% 6.00% 2.35s 2.49s detect_fts (datasette/utils/init.py:571) 4.00% 4.00% 1.34s 1.34s _write_to_self (asyncio/selector_events.py:140) ``` Relevant code: https://github.com/simonw/datasette/blob/798f075ef9b98819fdb564f9f79c78975a0f71e8/datasette/utils/init.py#L609-L625

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1673/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1174697144 I_kwDOBm6k_c5GBHS4 1672 Refactor CSV handling code out of DataView simonw 9599 open 0   Datasette 1.0 3268330 1 2022-03-20T21:47:00Z 2022-03-20T21:52:39Z   OWNER  

I think the way to get rid of most of the remaining complexity in DataView is to refactor how CSV stuff works - pulling it in line with other export factors and extracting the streaming mechanism. Opening a fresh issue for that.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1660#issuecomment-1073355032

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1672/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
688351054 MDU6SXNzdWU2ODgzNTEwNTQ= 140 Idea: insert-files mechanism for adding extra columns with fixed values simonw 9599 open 0     1 2020-08-28T20:57:36Z 2022-03-20T19:45:45Z   OWNER  

Say for example you want to populate a file_type column with the value gif. That could work like this:

sqlite-utils insert-files gifs.db images *.gif \ -c path -c md5 -c last_modified:mtime \ -c file_type:text:gif --pk=path So a column defined as a text column with a value that follows a second colon.

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/140/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1149310456 I_kwDOBm6k_c5EgRX4 1641 Tweak mobile keyboard settings simonw 9599 open 0     1 2022-02-24T13:47:10Z 2022-02-24T13:49:26Z   OWNER  

https://developer.apple.com/library/archive/documentation/StringsTextFonts/Conceptual/TextAndWebiPhoneOS/KeyboardManagement/KeyboardManagement.html#//apple_ref/doc/uid/TP40009542-CH5-SW12

autocorrect="off" is worth experimenting with.

Twitter: https://twitter.com/forestgregg/status/1496842959563726852

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1641/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1122446693 I_kwDOCGYnMM5C5y1l 394 Test against Python 3.11-dev simonw 9599 open 0     1 2022-02-02T22:21:03Z 2022-02-03T21:06:35Z   OWNER  

Same as: - https://github.com/simonw/datasette/issues/1621

sqlite-utils 140912432 issue    
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/394/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1113384383 I_kwDOBm6k_c5CXOW_ 1611 Avoid ever running count(*) against SpatiaLite KNN table simonw 9599 open 0     1 2022-01-25T03:32:54Z 2022-02-02T06:45:47Z   OWNER  

Got this in a trace:

Looks like running count(*) against KNN took 83s! It ignored the time limit. And still only returned a count of 0.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1611/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
328155946 MDU6SXNzdWUzMjgxNTU5NDY= 301 --spatialite option for "datasette publish heroku" simonw 9599 open 0     1 2018-05-31T14:13:09Z 2022-01-20T21:28:50Z   OWNER  

Split off from #243. Need to figure out how to install and configure SpatiaLite on Heroku.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/301/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
849975810 MDU6SXNzdWU4NDk5NzU4MTA= 1292 Research ctypes.util.find_library('spatialite') simonw 9599 open 0     1 2021-04-04T22:36:59Z 2022-01-20T21:28:50Z   OWNER  

Spotted this in the Django SpatiaLite backend: https://github.com/django/django/blob/8f6a7a0e9e7c5404af6520ae606927e32415eb00/django/contrib/gis/db/backends/spatialite/base.py#L24-L36

python ctypes.util.find_library('spatialite')

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1292/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1108300685 I_kwDOBm6k_c5CD1ON 1604 Option to assign a domain/subdomain using `datasette publish cloudrun` simonw 9599 open 0     1 2022-01-19T16:21:17Z 2022-01-19T16:23:54Z   OWNER  

Looks like this API should be able to do that: https://twitter.com/steren/status/1483835859191304192 - https://cloud.google.com/run/docs/reference/rest/v1/namespaces.domainmappings/create

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1604/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
733999615 MDU6SXNzdWU3MzM5OTk2MTU= 1079 Handle long breadcrumbs better with new menu simonw 9599 open 0     1 2020-11-01T15:57:41Z 2022-01-13T22:21:29Z   OWNER  

On this page when signed in as root: https://latest.datasette.io/fixtures/roadside_attraction_characteristics/1

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1079/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
895686039 MDU6SXNzdWU4OTU2ODYwMzk= 1336 Document turning on WAL for live served SQLite databases simonw 9599 open 0     1 2021-05-19T17:08:58Z 2022-01-13T21:55:59Z   OWNER  

Datasette docs don't talk about WAL yet, which allows you to safely serve reads from a database file while it is accepting writes.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1336/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1100499619 I_kwDOBm6k_c5BmEqj 1592 Row pages should show links to foreign keys simonw 9599 open 0     1 2022-01-12T15:50:20Z 2022-01-12T15:52:17Z   OWNER  

Refs #1518 refactor.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1592/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1091838742 I_kwDOBm6k_c5BFCMW 1585 Fire base caching for `publish cloudrun` simonw 9599 open 0     1 2022-01-01T15:38:15Z 2022-01-01T15:40:38Z   OWNER  

https://gist.github.com/steren/03d3e58c58c9a53fd49bb78f58541872 has a recipe for this, via https://twitter.com/steren/status/1477038411114446848

Could this enable easier vanity URLs of the format https://$project_id.web.app/? How about CDN caching?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1585/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1084185188 I_kwDOBm6k_c5An1pk 1573 Make trace() a documented internal API simonw 9599 open 0     1 2021-12-19T20:32:56Z 2021-12-19T21:13:13Z   OWNER  

This should be documented so plugin authors can use it to add their own custom traces: https://github.com/simonw/datasette/blob/8f311d6c1d9f73f4ec643009767749c17b5ca5dd/datasette/tracer.py#L28-L52

Including the new kwargs pattern I added in #1571: https://github.com/simonw/datasette/blob/f65817000fdf87ce8a0c23edc40784ebe33b5842/datasette/database.py#L128-L132

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1573/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
705840673 MDU6SXNzdWU3MDU4NDA2NzM= 972 Support faceting against arbitrary SQL queries simonw 9599 open 0     1 2020-09-21T19:00:43Z 2021-12-15T18:02:20Z   OWNER  

... support for running facets against arbitrary custom SQL queries is half-done in that facets now execute against wrapped subqueries as-of ea66c45df96479ef66a89caa71fff1a97a862646

https://github.com/simonw/datasette/blob/ea66c45df96479ef66a89caa71fff1a97a862646/datasette/facets.py#L192-L200 Originally posted by @simonw in https://github.com/simonw/datasette/issues/971#issuecomment-696307922

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/972/reactions",
    "total_count": 3,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 3,
    "rocket": 0,
    "eyes": 0
}
   
1071071397 I_kwDODFdgUs4_10Cl 69 View that combines issues and issue comments simonw 9599 open 0     1 2021-12-04T00:34:33Z 2021-12-04T00:34:52Z   MEMBER  

I want to see a reverse chronologically ordered interface onto both issues and comments - essentially a unified log of comments and issues opened across one or multiple projects.

github-to-sqlite 207052882 issue    
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/69/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1069881276 I_kwDOBm6k_c4_xRe8 1541 Different default layout for row page simonw 9599 open 0     1 2021-12-02T18:56:36Z 2021-12-02T18:56:54Z   OWNER  

The row page displays as a table even though it only has one table row.

maybe default to the same display as the narrow page version, even for wide pages?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1541/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1058803238 I_kwDOBm6k_c4_HA4m 1520 Pattern for avoiding accidental URL over-rides simonw 9599 open 0     1 2021-11-19T18:28:05Z 2021-11-19T18:29:26Z   OWNER  

Following #1517 I'm experimenting with a plugin that does this: python @hookimpl def register_routes(): return [ (r"/(?P<db_name>[^/]+)/(?P<table_and_format>[^/]+?)$", Table().view), ] This is supposed to replace the default table page with new code... but there's a problem: /-/versions on that instance now returns 404 Database '-' does not exist!

Need to figure out a pattern to avoid that happening. Plugins get to add their routes before Datasette's default routes, which is why this is happening here.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1520/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
812704869 MDU6SXNzdWU4MTI3MDQ4Njk= 1237 ?_pretty=1 option for pretty-printing JSON output simonw 9599 open 0   Datasette 1.0 3268330 1 2021-02-20T20:54:40Z 2021-11-16T18:28:33Z   OWNER  

Suggested by @frankieroberto in https://github.com/simonw/datasette/issues/782#issuecomment-782746755

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1237/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1052247023 I_kwDOBm6k_c4-uAPv 1505 Datasette should have an option to output CSV with semicolons simonw 9599 open 0     1 2021-11-12T18:02:21Z 2021-11-16T11:40:52Z   OWNER     datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1505/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
1054246919 I_kwDOBm6k_c4-1ogH 1511 Review plugin hooks for Datasette 1.0 simonw 9599 open 0   Datasette 1.0 3268330 1 2021-11-15T23:26:05Z 2021-11-16T01:20:14Z   OWNER  

I need to perform a detailed review of the plugin interface - especially the plugin hooks like register_facet_classes() which I don't yet have complete confidence in.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1511/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
990367646 MDU6SXNzdWU5OTAzNjc2NDY= 1462 Separate out "debug" options from "root" options simonw 9599 open 0     1 2021-09-07T21:27:34Z 2021-09-07T21:34:33Z   OWNER  

I ditched "root" for "admin" because root by default gives you a whole bunch of stuff which I think could be confusing:

Maybe the real problem here is that I'm conflating "root" permissions with "debug" options. Perhaps there should be an extra Datasette mode that unlocks debug tools for the root user?

Originally posted by @simonw in https://github.com/simonw/datasette-app-support/issues/8#issuecomment-914638998

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1462/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
970463436 MDExOlB1bGxSZXF1ZXN0NzEyNDEyODgz 1434 Enrich arbitrary query results with foreign key links and column descriptions simonw 9599 open 0     1 2021-08-13T14:43:01Z 2021-08-19T21:18:58Z   OWNER simonw/datasette/pulls/1434

Refs #1293, follows #942.

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1434/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
947044667 MDU6SXNzdWU5NDcwNDQ2Njc= 1398 Documentation on using Datasette as a library simonw 9599 open 0     1 2021-07-18T14:15:27Z 2021-07-30T03:21:49Z   OWNER  

Instantiating Datasette() directly is an increasingly interesting pattern. I do it in tests all the time, but thanks to datasette.client there are plenty of neat things you can do with it in a library context.

Maybe support from datasette import Datasette for this.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1398/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
776128565 MDU6SXNzdWU3NzYxMjg1NjU= 1163 "datasette insert data.db url-to-csv" simonw 9599 open 0     1 2020-12-29T23:21:21Z 2021-06-17T18:12:32Z   OWNER  

Refs #1160 - get filesystem imports working first for #1162, then add import-from-URL.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1163/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
743359646 MDU6SXNzdWU3NDMzNTk2NDY= 1096 TSV should be a default export option simonw 9599 open 0     1 2020-11-15T22:24:02Z 2021-06-17T18:12:31Z   OWNER  

Refs #1095

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1096/reactions",
    "total_count": 3,
    "+1": 3,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
913900374 MDU6SXNzdWU5MTM5MDAzNzQ= 1369 Don't show foreign key IDs twice if no label simonw 9599 open 0     1 2021-06-07T19:47:02Z 2021-06-07T19:47:24Z   OWNER  

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1369/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
901009787 MDU6SXNzdWU5MDEwMDk3ODc= 1340 Research: Cell action menu (like column action but for individual cells) simonw 9599 open 0     1 2021-05-25T15:49:16Z 2021-05-26T18:59:58Z   OWNER  

Had an idea today that it might be useful to select an individual cell and say things like "show me all other rows with the same value" - maybe even a set of other menu options against cells as well.

Mocked up a show-on-hover ellipses demo using the CSS inspector:

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1340/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
817544251 MDU6SXNzdWU4MTc1NDQyNTE= 1245 Sticky table column headers would be useful, especially on the query page simonw 9599 open 0     1 2021-02-26T17:42:51Z 2021-04-02T20:53:35Z   OWNER  

Suggestion from office hours.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1245/reactions",
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
787173276 MDU6SXNzdWU3ODcxNzMyNzY= 1193 Research plugin hook for alternative database backends simonw 9599 open 0     1 2021-01-15T20:27:50Z 2021-03-12T01:01:54Z   OWNER  

I started exploring what Datasette would like running against PostgreSQL in #670 and @dazzag24 did some work on Parquet described in #657.

I had initially thought this was WAY too much additional complexity, but I'm beginning to think that the Database class may be small enough that having it abstract away the details of running queries against alternative database backends could be feasible.

A bigger issue is SQL generation, but I realized that most of Datasette's SQL generation code exists just in the TableView class that runs the table page. If this was abstracted into some kind of SQL builder that could be then customized per-database it might be reasonable to get it working.

Very unlikely for this to make it into Datasette 1.0, but maybe this would be the defining feature of Datasette 2.0?

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1193/reactions",
    "total_count": 3,
    "+1": 3,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
821841046 MDU6SXNzdWU4MjE4NDEwNDY= 6 Upgrade to latest sqlite-utils simonw 9599 open 0     1 2021-03-04T07:21:54Z 2021-03-04T07:22:51Z   MEMBER  

This is pinned to v1 at the moment.

google-takeout-to-sqlite 206649770 issue    
{
    "url": "https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/6/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
811505638 MDU6SXNzdWU4MTE1MDU2Mzg= 1234 Runtime support for ATTACHing multiple databases simonw 9599 open 0     1 2021-02-18T22:06:47Z 2021-02-22T21:06:28Z   OWNER  

The implementation in #1232 is ready to land. It's the simplest-thing-that-could-possibly-work: you can run datasette one.db two.db three.db --crossdb and then use the /_memory page to run joins across tables from multiple databases.

It only works on the first 10 databases that were passed to the command-line. This means that if you have a Datasette instance with hundreds of attached databases (see Datasette Library) this won't be particularly useful for you.

So... a better, future version of this feature would be one that lets you join across databases on command - maybe by hitting /_memory?attach=db1&attach=db2 to get a special connection.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/283#issuecomment-781665560

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1234/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
811458446 MDU6SXNzdWU4MTE0NTg0NDY= 1233 "datasette publish cloudrun" cannot publish files with spaces in their name simonw 9599 open 0     1 2021-02-18T21:08:31Z 2021-02-18T21:10:08Z   OWNER  

Got this error: ``` Step 6/9 : RUN datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json ---> Running in db9da0068592 Usage: datasette inspect [OPTIONS] [FILES]... Try 'datasette inspect --help' for help.

Error: Invalid value for '[FILES]...': Path 'extra' does not exist. The command '/bin/sh -c datasette inspect fixtures.db extra database.db --inspect-file inspect-data.json' returned a non-zero code: 2 ERROR ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 2 While working on the demo for #1232, using this deploy command: GITHUB_SHA=crossdb datasette publish cloudrun fixtures.db 'extra database.db' \ -m fixtures.json \ --plugins-dir=plugins \ --branch=$GITHUB_SHA \ --version-note=$GITHUB_SHA \ --extra-options="--setting template_debug 1 --crossdb" \ --install=pysqlite3-binary \ --service=datasette-latest-crossdb ```

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1233/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
778682317 MDU6SXNzdWU3Nzg2ODIzMTc= 1173 GitHub Actions workflow to build manylinux binary simonw 9599 open 0     1 2021-01-05T07:41:11Z 2021-01-05T07:41:43Z   OWNER  

Refs #1171 and #93

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1173/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
776101101 MDU6SXNzdWU3NzYxMDExMDE= 1161 Update a whole bunch of links to datasette.io instead of datasette.readthedocs.io simonw 9599 open 0     1 2020-12-29T21:47:31Z 2020-12-29T21:49:57Z   OWNER  

https://ripgrep.datasette.io/-/ripgrep?pattern=%28datasette%7Csqlite-utils%29%5C.readthedocs%5C.io

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1161/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
463492815 MDU6SXNzdWU0NjM0OTI4MTU= 534 500 error on m2m facet detection simonw 9599 open 0     1 2019-07-03T00:42:42Z 2020-12-17T05:08:22Z   OWNER  

This may help debug: diff --git a/datasette/facets.py b/datasette/facets.py index 76d73e5..07a4034 100644 --- a/datasette/facets.py +++ b/datasette/facets.py @@ -499,11 +499,14 @@ class ManyToManyFacet(Facet): "outgoing" ] if len(other_table_outgoing_foreign_keys) == 2: - destination_table = [ - t - for t in other_table_outgoing_foreign_keys - if t["other_table"] != self.table - ][0]["other_table"] + try: + destination_table = [ + t + for t in other_table_outgoing_foreign_keys + if t["other_table"] != self.table + ][0]["other_table"] + except IndexError: + import pdb; pdb.pm() # Only suggest if it's not selected already if ("_facet_m2m", destination_table) in args: continue

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/534/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
765637324 MDU6SXNzdWU3NjU2MzczMjQ= 1144 JavaScript to help plugins interact with the fragment part of the URL simonw 9599 open 0     1 2020-12-13T20:36:06Z 2020-12-14T14:47:11Z   OWNER  

Suggested by Markus Holtermann on Twitter, who is building https://github.com/MarkusH/datasette-chartjs

I've been looking at datasette-vega for how you persist chart settings between form submissions. I've adopted that for datasette-chartjs. Any thoughts on adding a public JS API to #datasette itself, that plugins can rely on?

I'm talking about functions like onFragmentChange, serialize, unserialize, ... That turn an object into a URL encoded string and put it into the location's hash. And also updating all links/forms automatically.

Essentially, a plugins could do something like document.datasette.setConfigValue('prefix', 'foo', 'bar') and .getConfigValue('prefix', 'foo'). And the functions would take care of updating document.location.hash, all (necessary) a.href and form.action

https://twitter.com/m_holtermann/status/1338183973311295492

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1144/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
753000405 MDU6SXNzdWU3NTMwMDA0MDU= 53 Command for fetching file contents simonw 9599 open 0     1 2020-11-29T20:31:04Z 2020-11-30T00:36:09Z   MEMBER  

Something like this:

github-to-sqlite files github.db simonw/datasette

This would fetch all files from the main branch into a files table.

Additional options could handle things like pulling files from a branch or tag, or just pulling files that match a specific glob or that exist in a specific directory.

github-to-sqlite 207052882 issue    
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/53/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
718934942 MDU6SXNzdWU3MTg5MzQ5NDI= 1 Documentation on how to use this with Datasette simonw 9599 open 0     1 2020-10-11T21:56:27Z 2020-10-11T22:14:00Z   MEMBER  

In particular how to use datasette-render-images to see the images.

evernote-to-sqlite 303218369 issue    
{
    "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/1/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
718910318 MDU6SXNzdWU3MTg5MTAzMTg= 1015 Research: could Datasette install its own plugins? simonw 9599 open 0     1 2020-10-11T19:33:06Z 2020-10-11T19:35:04Z   OWNER  

It would be cool if Datasette could offer a plugin browsing interface where users could install plugins by clicking "Install" on them - similar to how VS Code extensions work.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1015/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
718272593 MDU6SXNzdWU3MTgyNzI1OTM= 1007 set-env and add-path commands have been deprecated simonw 9599 open 0     1 2020-10-09T16:21:18Z 2020-10-09T16:23:51Z   OWNER  

https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

Starting today runner version 2.273.5 will begin to warn you if you use the add-path or set-env commands. We are monitoring telemetry for the usage of these commands and plan to fully disable them in the future.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1007/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
626211658 MDU6SXNzdWU2MjYyMTE2NTg= 778 Ability to configure keyset pagination for views and queries simonw 9599 open 0     1 2020-05-28T04:48:56Z 2020-10-02T02:26:25Z   OWNER  

Currently views offer pagination, but it uses offset/limit - e.g. https://latest.datasette.io/fixtures/paginated_view?_next=100

This means pagination will perform poorly on deeper pages.

If a view is based on a table that has a primary key it should be possible to configure efficient keyset pagination that works the same way that table pagination works.

This may be as simple as configuring a column that can be treated as a "primary key" for the purpose of pagination using metadata.json - or with a ?_view_pk=colname querystring argument.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/778/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
712368432 MDU6SXNzdWU3MTIzNjg0MzI= 984 Review accessibility of new column action menus simonw 9599 open 0     1 2020-09-30T23:56:44Z 2020-10-01T00:01:36Z   OWNER  

Feature added in #981

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/984/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
694136490 MDU6SXNzdWU2OTQxMzY0OTA= 15 Add a bunch of config examples simonw 9599 open 0     1 2020-09-05T17:58:43Z 2020-09-18T23:17:39Z   MEMBER  

I can bring these over from my personal Dogsheep.

dogsheep-beta 197431109 issue    
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-beta/issues/15/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
503243784 MDU6SXNzdWU1MDMyNDM3ODQ= 3 Extract images into separate tables simonw 9599 open 0     1 2019-10-07T05:43:01Z 2020-09-01T06:17:45Z   MEMBER  

As already done with authors. Slightly harder because images do not have a universally unique ID. Also need to figure out what to do about there being columns for both image and images.

pocket-to-sqlite 213286752 issue    
{
    "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/3/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
687694947 MDU6SXNzdWU2ODc2OTQ5NDc= 954 Remove old register_output_renderer dict mechanism in Datasette 1.0 simonw 9599 open 0   Datasette 1.0 3268330 1 2020-08-28T04:04:23Z 2020-08-28T04:56:31Z   OWNER  

Documentation says that the old dictionary mechanism will be deprecated by 1.0:

https://github.com/simonw/datasette/blob/799ecae94824640bdff21f86997f69844048d5c3/docs/plugin_hooks.rst#L460 Originally posted by @simonw in https://github.com/simonw/datasette/issues/953#issuecomment-682312494

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/954/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
643510821 MDU6SXNzdWU2NDM1MTA4MjE= 862 Set an upper limit on total facet suggestion time for a page simonw 9599 open 0     1 2020-06-23T03:57:55Z 2020-06-23T03:58:48Z   OWNER  

If a table has 100 columns the facet suggestion code will currently run 100 times, taking a max of facet_suggest_time_limit_ms which defaults to 50ms per column:

https://github.com/simonw/datasette/blob/000528192eaf891118932250141dabe7a1561ece/datasette/facets.py#L142-L162

So for 100 columns, that's 100 * 50ms = 5s total time that might be spent attempting to calculate facets on a large table!

I should implement a hard upper limit on the total amount of time taken suggesting facets - probably of around 500ms. If it takes longer than that the remaining columns will not be considered.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/862/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
574021194 MDU6SXNzdWU1NzQwMjExOTQ= 691 --reload sould reload server if code in --plugins-dir changes simonw 9599 open 0     1 2020-03-02T14:42:21Z 2020-06-14T02:35:17Z   OWNER  
datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/691/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
621323348 MDU6SXNzdWU2MjEzMjMzNDg= 24 Configurable URL for images simonw 9599 open 0     1 2020-05-19T22:25:56Z 2020-05-20T06:00:29Z   MEMBER  

This is hard-coded at the moment, which is bad: https://github.com/dogsheep/photos-to-sqlite/blob/d5d69b9019703c47bc251444838578dd752801e2/photos_to_sqlite/cli.py#L269-L272

dogsheep-photos 256834907 issue    
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/24/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
616087149 MDU6SXNzdWU2MTYwODcxNDk= 765 publish heroku should default to currently tagged version simonw 9599 open 0     1 2020-05-11T18:24:06Z 2020-05-11T18:25:43Z   OWNER  

Had a report that deploying to Heroku was using the previously installed version of Datasette, not the latest.

Could be because of this:

https://github.com/simonw/datasette/blob/af6c6c5d6f929f951c0e63bfd1c82e37a071b50f/datasette/publish/heroku.py#L172-L179

Heroku documentation recommends pinning to specific versions https://devcenter.heroku.com/articles/python-pip

So... we could ensure we default to an install value of ["datasette>=current_tag"].

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/765/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
612860758 MDU6SXNzdWU2MTI4NjA3NTg= 18 Switch CI solution to GitHub Actions with a macOS runner simonw 9599 open 0     1 2020-05-05T20:03:50Z 2020-05-05T23:49:18Z   MEMBER  

Refs #17.

dogsheep-photos 256834907 issue    
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/18/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
606033104 MDU6SXNzdWU2MDYwMzMxMDQ= 12 If less than 500MB, show size in MB not GB simonw 9599 open 0     1 2020-04-24T04:35:01Z 2020-04-24T04:35:25Z   MEMBER  

Just saw this: Uploading 0.05 GB

dogsheep-photos 256834907 issue    
{
    "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/12/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
602619330 MDU6SXNzdWU2MDI2MTkzMzA= 45 Use raise_for_status() everywhere simonw 9599 open 0     1 2020-04-19T04:38:28Z 2020-04-19T04:39:22Z   MEMBER  

I keep seeing errors which I think are caused by authentication or rate limit problems but which appear to be unexpected JSON responses - presumably because they are actually an error message.

Recent example: https://github.com/simonw/jsk-fellows-on-twitter/runs/598892575

Using response.raise_for_status() everywhere will make these errors less confusing.

twitter-to-sqlite 206156866 issue    
{
    "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/45/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
574035432 MDU6SXNzdWU1NzQwMzU0MzI= 692 is_hidden_table context variable on table.html page simonw 9599 open 0     1 2020-03-02T15:03:25Z 2020-03-02T15:03:48Z   OWNER  

It's useful to know if a table is hidden when rendering that page. datasette-configure-fts for example may want to disallow enabling search on hidden tables.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/692/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
559964149 MDU6SXNzdWU1NTk5NjQxNDk= 665 Introduce a SQL statement parser in Python simonw 9599 open 0     1 2020-02-04T20:36:05Z 2020-02-04T20:36:48Z   OWNER  

254 and #653 are both examples of problems that could be solved using a real SQL parser in Python.

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/665/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
501773982 MDExOlB1bGxSZXF1ZXN0MzIzOTgzNzMy 579 New connection pooling simonw 9599 open 0     1 2019-10-02T23:22:19Z 2019-11-15T22:57:21Z   OWNER simonw/datasette/pulls/579

See #569

datasette 107914493 pull    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/579/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
0  
516874735 MDU6SXNzdWU1MTY4NzQ3MzU= 613 Basic join support for table view simonw 9599 open 0     1 2019-11-03T19:12:53Z 2019-11-03T19:14:01Z   OWNER  

I think it would be possible to support basic foreign key joins on the table page.

The user could specify columns that should result in a join (from a set of suggestions similar to how facets work right now) and they could then be passed as ?_join=city_id arguments.

This feature will make a lot of sense when combined with the ability to show / hide / customize columns, see #292

datasette 107914493 issue    
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/613/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   
488874815 MDU6SXNzdWU0ODg4NzQ4MTU= 5 Write tests that simulate the Twitter API simonw 9599 open 0     1 2019-09-03T23:55:35Z 2019-09-03T23:56:28Z   MEMBER  

I can use betamax for this: https://pypi.org/project/betamax/

twitter-to-sqlite 206156866 issue    
{
    "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/5/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);
Powered by Datasette · Queries took 570.736ms · About: github-to-sqlite
  • Sort ascending
  • Sort descending
  • Facet by this
  • Hide this column
  • Show all columns
  • Show not-blank rows