issues

Table actions

312 rows where comments = 2 and user = 9599 sorted by updated_at descending

✖
✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: milestone, author_association, draft, created_at (date), updated_at (date), closed_at (date)

type 2

state 2

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1570375808 I_kwDODFdgUs5dmgiA 79 Deploy demo job is failing due to rate limit simonw 9599 open 0     2 2023-02-03T20:05:01Z 2023-12-08T14:50:15Z   MEMBER   github-to-sqlite 207052882 issue     
{
    "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/79/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
2029161033 I_kwDOCGYnMM548opJ 606 str and int as aliases for text and integer simonw 9599 closed 0     2 2023-12-06T18:35:49Z 2023-12-06T19:44:04Z 2023-12-06T18:49:32Z OWNER  

I keep making this mistake: bash sqlite-utils add-column content.db assets _since int ``` Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME [[integer|float|b lob|text|INTEGER|FLOAT|BLOB|TEXT]] Try 'sqlite-utils add-column -h' for help.

Error: Invalid value for '[[integer|float|blob|text|INTEGER|FLOAT|BLOB|TEXT]]': 'int' is not one of 'integer', 'float', 'blob', 'text', 'INTEGER', 'FLOAT', 'BLOB', 'TEXT'. ```

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/606/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1898927976 I_kwDOBm6k_c5xL1do 2186 Mechanism for register_output_renderer hooks to access full count simonw 9599 open 0   Datasette 1.0 3268330 2 2023-09-15T18:57:54Z 2023-09-15T19:27:59Z   OWNER  

The cause of this bug: - https://github.com/simonw/datasette-export-notebook/issues/17

Is that datasette-export-notebook was consulting data["filtered_table_rows_count"] in the render output plugin function in order to show the total number of rows that would be exported.

That field is no longer available by default - the "count" field is only available if ?_extra=count was passed.

It would be useful if plugins like this could access the total count on demand, should they need to.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2186/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1891614971 I_kwDOCGYnMM5wv8D7 594 Represent compound foreign keys in table.foreign_keys output simonw 9599 open 0     2 2023-09-12T03:48:24Z 2023-09-12T03:51:13Z   OWNER  

Given this schema: sql CREATE TABLE departments ( campus_name TEXT NOT NULL, dept_code TEXT NOT NULL, dept_name TEXT, PRIMARY KEY (campus_name, dept_code) ); CREATE TABLE courses ( course_code TEXT PRIMARY KEY, course_name TEXT, campus_name TEXT NOT NULL, dept_code TEXT NOT NULL, FOREIGN KEY (campus_name, dept_code) REFERENCES departments(campus_name, dept_code) ); The output of db["courses"].foreign_keys right now is: [ForeignKey(table='courses', column='campus_name', other_table='departments', other_column='campus_name'), ForeignKey(table='courses', column='dept_code', other_table='departments', other_column='dept_code')] Which suggests two normal foreign keys, not one compound foreign key.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/594/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1846076261 I_kwDOBm6k_c5uCONl 2139 border-color: ##ff0000 bug - two hashes simonw 9599 closed 0   Datasette 1.0a-next 8755003 2 2023-08-11T01:22:58Z 2023-08-11T05:16:24Z 2023-08-11T05:16:24Z OWNER  

Spotted this on https://latest.datasette.io/extra_database

```html

```
 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2139/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1844213115 I_kwDOBm6k_c5t7HV7 2138 on_success_message_sql option for writable canned queries simonw 9599 closed 0   Datasette 1.0a-next 8755003 2 2023-08-10T00:20:14Z 2023-08-10T00:39:40Z 2023-08-10T00:34:26Z OWNER  

Or... how about if the on_success_message option could define a SQL query to be executed to generate that message? Maybe on_success_message_sql.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2138/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1840329615 I_kwDOBm6k_c5tsTOP 2130 Render plugin mechanism needs `error` and `truncated` fields simonw 9599 closed 0   Datasette 1.0a3 9700784 2 2023-08-07T23:19:19Z 2023-08-08T01:51:54Z 2023-08-08T01:47:42Z OWNER  

While working on: - https://github.com/simonw/datasette/pull/2118

It became clear that the render callback function documented here: https://docs.datasette.io/en/0.64.3/plugin_hooks.html#register-output-renderer-datasette

Needs to grow the ability to be told if an error occurred (an error string) and if the results were truncated (a truncated boolean).

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2130/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1816997390 I_kwDOCGYnMM5sTS4O 576 Backfill the release notes prior to 0.4 simonw 9599 closed 0     2 2023-07-23T05:41:42Z 2023-07-23T05:49:51Z 2023-07-23T05:48:21Z OWNER  

Currently the changelog starts at 0.4:

https://sqlite-utils.datasette.io/en/3.34/changelog.html#id115

I want the other releases - according to https://pypi.org/project/sqlite-utils/#history there are three missing:

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/576/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1816919568 I_kwDOCGYnMM5sS_4Q 575 Python API ability to opt-out of connection plugins simonw 9599 closed 0     2 2023-07-22T23:01:13Z 2023-07-22T23:17:22Z 2023-07-22T23:08:22Z OWNER  

Plugins affecting the CLI by default makes sense to me.

I'm less confident about them always affecting users of the Python API.

I'm going to have them apply by default, but I'm going to add a mechanism to opt-out on an individual database basis. Basically this:

```python from sqlite_utils import Database db = Database(memory=True, execute_plugins=False)

Anything using db from here on will not execute plugins

``` cc @asg017

Refs: - #567 - #574

sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/575/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1816877910 I_kwDOCGYnMM5sS1tW 572 Don't test Python 3.7 against textual simonw 9599 closed 0     2 2023-07-22T19:57:03Z 2023-07-22T22:16:50Z 2023-07-22T22:16:50Z OWNER  

Spotted this in the GitHub Actions logs:

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/572/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1808215339 I_kwDOBm6k_c5rxy0r 2104 Tables starting with an underscore should be treated as hidden simonw 9599 open 0     2 2023-07-17T17:13:53Z 2023-07-18T22:41:37Z   OWNER  

Plugins can then take advantage of this pattern, for example: - https://github.com/simonw/datasette-auth-tokens/pull/8

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2104/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1765870617 I_kwDOBm6k_c5pQQwZ 2087 `--settings settings.json` option simonw 9599 open 0     2 2023-06-20T17:48:45Z 2023-07-14T17:02:03Z   OWNER  

https://discord.com/channels/823971286308356157/823971286941302908/1120705940728066080

May I add a request to the whole metadata / settings ? Allow to pass --settings path/to/settings.json instead of having to rely exclusively on directory mode to centralize settings (this would reflect the behavior of providing metadata)

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2087/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1795187493 I_kwDODLZ_YM5rAGMl 12 Switch to pyproject.toml simonw 9599 closed 0     2 2023-07-09T01:06:56Z 2023-07-09T01:19:43Z 2023-07-09T01:19:42Z MEMBER   pocket-to-sqlite 213286752 issue     
{
    "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1781047747 I_kwDOBm6k_c5qKKHD 2092 test_homepage intermittent failure simonw 9599 closed 0     2 2023-06-29T15:20:37Z 2023-06-29T15:26:28Z 2023-06-29T15:24:13Z OWNER  

e.g. in https://github.com/simonw/datasette/actions/runs/5413590227/jobs/9839373852

``` =================================== FAILURES =================================== _____ testhomepage _______ [gw0] linux -- Python 3.7.17 /opt/hostedtoolcache/Python/3.7.17/x64/bin/python

ds_client = <datasette.app.DatasetteClient object at 0x7f85d271ef50>

@pytest.mark.asyncio
async def test_homepage(ds_client):
    response = await ds_client.get("/.json")
    assert response.status_code == 200
    assert "application/json; charset=utf-8" == response.headers["content-type"]
    data = response.json()
    assert data.keys() == {"fixtures": 0}.keys()
    d = data["fixtures"]
    assert d["name"] == "fixtures"
    assert d["tables_count"] == 24
    assert len(d["tables_and_views_truncated"]) == 5
    assert d["tables_and_views_more"] is True
    # 4 hidden FTS tables + no_primary_key (hidden in metadata)
    assert d["hidden_tables_count"] == 6
    # 201 in no_primary_key, plus 6 in other hidden tables:

  assert d["hidden_table_rows_sum"] == 207, data

E AssertionError: {'fixtures': {'color': '9403e5', 'hash': None, 'hidden_table_rows_sum': 0, 'hidden_tables_count': 6, ...}} E assert 0 == 207 ``` My guess is that this is a timing error, where very occasionally the "count rows but stop counting if it exceeds a time limit" thing fails.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2092/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1781005740 I_kwDOBm6k_c5qJ_2s 2090 Adopt ruff for linting simonw 9599 open 0     2 2023-06-29T14:56:43Z 2023-06-29T15:05:04Z   OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2090/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1718635018 PR_kwDOCGYnMM5Q9lY4 553 Reformatted CLI examples in docs simonw 9599 closed 0     2 2023-05-21T20:44:34Z 2023-05-21T20:57:27Z 2023-05-21T20:57:23Z OWNER simonw/sqlite-utils/pulls/553

Refs: - #551

:books: Documentation preview :books:: https://sqlite-utils--553.org.readthedocs.build/en/553/

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/553/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1718515590 I_kwDOCGYnMM5mbneG 544 New options for analyze-tables --common-limit --no-most and --no-least simonw 9599 closed 0     2 2023-05-21T14:03:19Z 2023-05-21T17:03:06Z 2023-05-21T16:19:31Z OWNER  

The "least common" section is frequently uninteresting, especially for huge tables with a large number of repeated-once values.

sqlite-utils analyze-tables content.db repos --common-limit 20 --no-least
sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/544/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1718550688 PR_kwDOCGYnMM5Q9VH0 546 Analyze tables options: --common-limit, --no-most, --no-least simonw 9599 closed 0     2 2023-05-21T15:54:39Z 2023-05-21T16:19:30Z 2023-05-21T16:19:30Z OWNER simonw/sqlite-utils/pulls/546

Refs #544

  • [x] Documentation for CLI options
  • [x] Documentation for new Python API parameters: most_common: bool and least_common: bool
  • [x] Tests for CLI
  • [x] Tests for Python API
 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/546/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1708030220 I_kwDOBm6k_c5lznkM 2073 Faceting doesn't work against integer columns in views simonw 9599 open 0     2 2023-05-12T18:20:10Z 2023-05-12T18:24:07Z   OWNER  

Spotted this issue here: https://til.simonwillison.net/datasette/baseline

I had to do this workaround: sql create view baseline as select _key, spec, '' || json_extract(status, '$.is_baseline') as is_baseline, json_extract(status, '$.since') as baseline_since, json_extract(status, '$.support.chrome') as baseline_chrome, json_extract(status, '$.support.edge') as baseline_edge, json_extract(status, '$.support.firefox') as baseline_firefox, json_extract(status, '$.support.safari') as baseline_safari, compat_features, caniuse, usage_stats, status from [index] I think the core issue here is that, against a table, select * from x where integer_column = '1' works correctly, due to some kind of column type conversion mechanism... but this mechanism doesn't work against views.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2073/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1702354223 I_kwDOBm6k_c5ld90v 2070 Mechanism for deploying a preview of a branch using Vercel simonw 9599 closed 0     2 2023-05-09T16:21:45Z 2023-05-09T16:25:00Z 2023-05-09T16:24:31Z OWNER  

I prototyped that here: https://github.com/simonw/one-off-actions/blob/main/.github/workflows/deploy-datasette-branch-preview.yml

It deployed the json-extras-query branch here: https://datasette-preview-json-extras-query.vercel.app/

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2070/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1617602868 I_kwDOJHON9s5gaqk0 6 Character encoding problem simonw 9599 open 0     2 2023-03-09T16:44:34Z 2023-04-14T15:22:09Z   MEMBER  

I ran against a recent note with this in it:

Or just "Actions ⚙️ "

And got back:

Actions ⚙️

Pasting that into https://ftfy.vercel.app/?s=Actions+%E2%80%9A%C3%B6%C3%B4%C3%94%E2%88%8F%C3%A8+ gives this:

python s = 'Actions â\x80\x9aöôÃ\x94â\x88\x8fè' s = s.encode('latin-1') s = s.decode('utf-8') s = s.encode('macroman') s = s.decode('utf-8') print(s)

apple-notes-to-sqlite 611552758 issue     
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/6/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1633077183 I_kwDOBm6k_c5hVse_ 2041 Remove obsolete table POST code simonw 9599 closed 0   Datasette 1.0a-next 8755003 2 2023-03-21T01:01:40Z 2023-03-21T01:17:44Z 2023-03-21T01:17:43Z OWNER  

Spotted this in: - #1999

POST /db/table currently executes obsolete code for inserting a row - I replaced that with /db/table/-/insert in https://github.com/simonw/datasette/commit/6e788b49edf4f842c0817f006eb9d865778eea5e but forgot to remove the old code.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2041/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1620516340 I_kwDOCGYnMM5glx30 533 ReadTheDocs error: not all arguments converted during string formatting simonw 9599 closed 0     2 2023-03-12T21:21:05Z 2023-03-12T21:25:33Z 2023-03-12T21:25:33Z OWNER  

This came up as a failure running tests for: - #531

Traceback on https://readthedocs.org/projects/sqlite-utils/builds/19749348/

``` File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/docutils/parsers/rst/states.py", line 889, in interpreted nodes, messages2 = role_fn(role, rawsource, text, lineno, self) File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py", line 103, in role title = caption % part TypeError: not all arguments converted during string formatting

Exception occurred: File "/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py", line 103, in role title = caption % part TypeError: not all arguments converted during string formatting ```

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/533/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1618249044 I_kwDOBm6k_c5gdIVU 2038 Consider a `strict_templates` setting simonw 9599 open 0     2 2023-03-10T02:09:13Z 2023-03-10T02:11:06Z   OWNER  

A setting which turns on Jinja strict mode, so any templates that access undefined variables raise a hard error.

Prototype here: diff diff --git a/datasette/app.py b/datasette/app.py index 40416713..1428a3f0 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -200,6 +200,7 @@ SETTINGS = ( "Allow display of SQL trace debug information with ?_trace=1", ), Setting("base_url", "/", "Datasette URLs should use this base path"), + Setting("strict_templates", False, "Raise errors for undefined template variables"), ) _HASH_URLS_REMOVED = "The hash_urls setting has been removed, try the datasette-hashed-urls plugin instead" OBSOLETE_SETTINGS = { @@ -399,11 +400,14 @@ class Datasette: ), ] ) + env_extras = {} + if self.setting("strict_templates"): + env_extras["undefined"] = StrictUndefined self.jinja_env = Environment( loader=template_loader, autoescape=True, enable_async=True, - undefined=StrictUndefined, + **env_extras, ) self.jinja_env.filters["escape_css_string"] = escape_css_string self.jinja_env.filters["quote_plus"] = urllib.parse.quote_plus Explored this idea a bit in: - #1999

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2038/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1616429236 I_kwDOJHON9s5gWMC0 4 Support incremental updates simonw 9599 open 0     2 2023-03-09T05:14:00Z 2023-03-09T18:20:56Z   MEMBER  

Running this script can take several hours against a large notes database.

Would be neat if it could run against just the notes that have been modified since it last ran. Could pull the max updated date and then keep on looping until it finds one modified before then.

Problem is I don't actually know what order it iterates over the notes in.

 apple-notes-to-sqlite 611552758 issue     
{
    "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/4/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1612296210 I_kwDOBm6k_c5gGbAS 2033 `datasette install -r requirements.txt` simonw 9599 closed 0     2 2023-03-06T22:17:17Z 2023-03-06T22:54:52Z 2023-03-06T22:27:34Z OWNER  

Would be useful for cases where you want to install a whole set of plugins in one go, e.g. when running tutorials in GitHub Codespaces.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2033/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1186696202 I_kwDOBm6k_c5Gu4wK 1696 Show foreign key label when filtering simonw 9599 open 0     2 2022-03-30T16:18:54Z 2023-01-29T20:56:20Z   OWNER  

For example here:

3 corresponds to "Human Related: Other" - it would be neat to display this in this area of the page somehow.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1696/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1558644003 I_kwDOBm6k_c5c5wUj 2006 Teach `datasette publish` to pin to `datasette<1.0` in a 0.x release simonw 9599 open 0   Datasette 1.0 3268330 2 2023-01-26T19:17:40Z 2023-01-26T19:20:53Z   OWNER  

I just realized that when I ship Datasette 1.0 there may be automated deployments out there which could deploy the 1.0 version by accident, potentially breaking any customizations that aren't compatible with the 1.0 changes.

I can hopefully help avoid that by shipping one last entry in the 0.x series that ensures datasette publish pins to <1.0 when it installs Datasette itself.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/2006/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1551113681 I_kwDOBm6k_c5cdB3R 1998 `datasette --version` should also show the SQLite version simonw 9599 open 0     2 2023-01-20T16:11:30Z 2023-01-20T18:19:06Z   OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1998/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1524867951 I_kwDOBm6k_c5a46Nv 1980 "Cannot sort table by id" when sortable_columns is used simonw 9599 open 0     2 2023-01-09T03:21:33Z 2023-01-09T03:23:53Z   OWNER  

I had an instance with this in metadata.yml:

yaml databases: timezones: tables: timezones: sortable_columns: - tzid When I clicked on the "Apply" button here:

It sent me to /timezones/timezones?_sort=id&id__exact=133 with the error message:

500: Cannot sort table by id

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1980/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1515182998 I_kwDOBm6k_c5aT9uW 1970 Path "None" in _internal database table simonw 9599 closed 0     2 2022-12-31T18:51:05Z 2022-12-31T19:22:58Z 2022-12-31T18:52:49Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1970/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1501713288 I_kwDOBm6k_c5ZglOI 1963 0.63.3 bugfix release simonw 9599 closed 0     2 2022-12-18T02:48:15Z 2022-12-18T03:26:55Z 2022-12-18T03:26:55Z OWNER  

I'm going to ship a release which back-ports these two fixes:

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1963/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1495821607 I_kwDOBm6k_c5ZKG0n 1953 Release notes for Datasette 1.0a2 simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-14T06:26:40Z 2022-12-15T02:02:15Z 2022-12-15T02:01:08Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1953/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
855296937 MDU6SXNzdWU4NTUyOTY5Mzc= 1295 Errors should have links to further information simonw 9599 open 0     2 2021-04-11T12:39:12Z 2022-12-14T23:28:49Z   OWNER  

Inspired by this tweet: https://twitter.com/willmcgugan/status/1381186384510255104

While I am thinking about faqs. I’d also like to add short URLs to Rich exceptions.

I loath cryptic error messages, and I’ve created a fair few myself. In Rich I’ve tried to make them as plain English as possible. But...

would be great if every error message linked to a page that explains the error in detail and offers fixes.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1295/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1483320357 I_kwDOBm6k_c5Yaawl 1937 /db/-/create API should require insert-rows permission to use row: or rows: option simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-08T01:33:09Z 2022-12-14T20:21:26Z 2022-12-14T20:21:26Z OWNER  

Otherwise someone with create-table but noinsert-rows permission could abuse it to insert data.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1937/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1497288666 I_kwDOBm6k_c5ZPs_a 1956 Handle abbreviations properly in permission_allowed_actor_restrictions simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-14T19:54:21Z 2022-12-14T20:04:29Z 2022-12-14T20:04:28Z OWNER  

This code currently assumes abbreviations are:

pyton action_initials = "".join([word[0] for word in action.split("-")])

https://github.com/simonw/datasette/blob/1a3dcf494376e32f7cff110c86a88e5b0a3f3924/datasette/default_permissions.py#L182-L208

That's no longer correct, they are now registered by the new plugin hook: - #1939

datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1956/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1495241162 I_kwDOBm6k_c5ZH5HK 1950 Bad ?_sort returns a 500 error, should be a 400 simonw 9599 closed 0     2 2022-12-13T22:08:16Z 2022-12-13T22:23:22Z 2022-12-13T22:23:22Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1950/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1200649124 I_kwDOBm6k_c5HkHOk 1708 Datasette 1.0 alpha upcoming release notes simonw 9599 open 0   Datasette 1.0a-next 8755003 2 2022-04-11T22:57:12Z 2022-12-13T05:29:06Z   OWNER  

I'm going to try writing the release notes first, to see if that helps unblock me.

⚠️ Any release notes in this issue are a draft, and should not be treated as the real thing ⚠️

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1708/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1493339206 I_kwDOBm6k_c5ZAoxG 1946 `datasette --get` mechanism for sending tokens simonw 9599 closed 0   Datasette 1.0a2 8711695 2 2022-12-13T04:25:05Z 2022-12-13T04:36:57Z 2022-12-13T04:36:57Z OWNER  

For the tests for datasette create-token it would be useful if datasette --get had a mechanism for sending an Authorization: Bearer X header.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1855#issuecomment-1347731288

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1946/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1483250004 I_kwDOBm6k_c5YaJlU 1936 Fix /db/table/-/upsert in the API explorer simonw 9599 open 0   Datasette 1.0 3268330 2 2022-12-08T00:59:34Z 2022-12-08T01:36:02Z   OWNER  

Split from: - #1931 - #1878

This is a bit tricky because the code needs to figure out what the primary keys are for an item, and whether or not rowid should be included.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1936/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1469043836 I_kwDOBm6k_c5Xj9R8 1917 Don't allow writable API to edit the `_memory` database simonw 9599 closed 0   Datasette 1.0a1 7867486 2 2022-11-30T04:51:59Z 2022-11-30T05:07:56Z 2022-11-30T05:07:55Z OWNER  

It shows up on https://latest.datasette.io/-/api (once you are signed in as root) - but there's no point in creating tables in it because they likely won't persist from one request to the next, as it's not a shared named database.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1917/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1468519699 I_kwDOBm6k_c5Xh9UT 1911 `/db/-/create` should support creating tables with compound primary keys simonw 9599 closed 0   Datasette 1.0a0 8658075 2 2022-11-29T18:30:47Z 2022-11-29T18:50:58Z 2022-11-29T18:48:05Z OWNER  

Found myself needing this to write the tests for: - #1864

datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1911/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1455928469 I_kwDOBm6k_c5Wx7SV 1903 Refactor all error classes into a datasette.exceptions module simonw 9599 open 0   Datasette 1.0 3268330 2 2022-11-18T22:44:45Z 2022-11-20T22:35:01Z   OWNER  

While working on this issue: - #1896

I realized that Datasette has error classes scattered around a fair bit, including some in the datasette.utils.asgi module for some reason.

I should clean these up.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1903/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1447388809 I_kwDOBm6k_c5WRWaJ 1887 Add a confirm step to the drop table API simonw 9599 closed 0   Datasette 1.0a0 8658075 2 2022-11-14T04:59:53Z 2022-11-15T19:59:59Z 2022-11-14T05:18:51Z OWNER  

In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057

Added drop table API in: - #1874

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1887/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1431786951 I_kwDOBm6k_c5VV1XH 1876 SQL query should wrap on SQL interrupted screen simonw 9599 closed 0     2 2022-11-01T17:14:01Z 2022-11-01T17:22:33Z 2022-11-01T17:22:33Z OWNER  

Just saw this:

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1876/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1423182778 I_kwDOCGYnMM5U1Au6 505 Release sqlite-utils 3.30 simonw 9599 closed 0     2 2022-10-25T22:20:05Z 2022-10-25T22:41:26Z 2022-10-25T22:41:16Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/505/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1420055377 I_kwDOBm6k_c5UpFNR 1847 Both _local_metadata and _metadata_local? simonw 9599 closed 0     2 2022-10-24T01:43:08Z 2022-10-24T01:53:13Z 2022-10-24T01:53:13Z OWNER  

Spotted this in the debugger against the datasette object while running tests (pytest -k test_permissions_cascade to be exact):

(Pdb) [p for p in dir(self) if p.startswith('_') and '__' not in p] ['_actor', '_asset_urls', '_connected_databases', '_crumb_items', '_local_metadata', '_metadata', '_metadata_local', '_metadata_recursive_update', '_permission_checks', '_plugins', '_prepare_connection', '_refresh_schemas', '_refresh_schemas_lock', '_register_custom_units', '_register_renderers', '_root_token', '_routes', '_secret', '_settings', '_show_messages', '_startup_hook_calculation', '_startup_hook_fired', '_startup_invoked', '_threads', '_versions', '_write_messages_to_response']

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1847/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1397193691 I_kwDOBm6k_c5TR3vb 1832 __bool__ method on Results simonw 9599 closed 0     2 2022-10-05T04:18:12Z 2022-10-05T04:32:33Z 2022-10-05T04:32:33Z OWNER  

Wrote this code today: https://github.com/simonw/datasette-public/blob/1401bfae50e71c1dfd2bfb6954f2e86d5a7ab21b/datasette_public/init.py#L41 python results = await db.execute( "select 1 from _public_tables where table_name = ?", [table_name] ) if len(results): return True Would be nice if I could use if results there instead.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1832/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1393903845 I_kwDOBm6k_c5TFUjl 1828 word-wrap: anywhere resulting in weird display simonw 9599 closed 0     2 2022-10-02T21:25:03Z 2022-10-02T23:01:17Z 2022-10-02T23:01:17Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1828/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1122427321 I_kwDOBm6k_c5C5uG5 1624 Index page `/` has no CORS headers simonw 9599 open 0     2 2022-02-02T21:56:10Z 2022-09-28T16:54:22Z   OWNER  

Compare the following: ``` % curl -I 'https://latest.datasette.io/fixtures' HTTP/1.1 200 OK link: https://latest.datasette.io/fixtures.json; rel="alternate"; type="application/json+datasette" cache-control: max-age=5 referrer-policy: no-referrer access-control-allow-origin: * access-control-allow-headers: Authorization access-control-expose-headers: Link content-type: text/html; charset=utf-8 x-databases: _memory, _internal, fixtures, extra_database Date: Wed, 02 Feb 2022 21:55:49 GMT Server: Google Frontend Transfer-Encoding: chunked

% curl -I 'https://latest.datasette.io/'
HTTP/1.1 200 OK link: https://latest.datasette.io/.json; rel="alternate"; type="application/json+datasette" content-type: text/html; charset=utf-8 x-databases: _memory, _internal, fixtures, extra_database Date: Wed, 02 Feb 2022 21:55:52 GMT Server: Google Frontend Transfer-Encoding: chunked ```

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1624/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1386593843 I_kwDOCGYnMM5Spb4z 494 Document how to use Just simonw 9599 closed 0     2 2022-09-26T19:25:12Z 2022-09-26T19:32:36Z 2022-09-26T19:26:39Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/494/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1363765916 I_kwDOCGYnMM5RSWqc 483 `sqlite-utils install` command simonw 9599 closed 0     2 2022-09-06T20:13:55Z 2022-09-26T19:04:43Z 2022-09-26T18:57:15Z OWNER  

With the addition of --functions in: - #471

In addition to the existing convert command, there are now very good reasons to want to install additional packages into the same virtual environment as sqlite-utils itself, to allow them to be used with those features.

This isn't easy if you installed the tool with pipx or brew install sqlite-utils.

Datasette solved this problem with the datasette install command:

sqlite-utils could benefit from the same idea.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/483/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
520508502 MDU6SXNzdWU1MjA1MDg1MDI= 31 "friends" command (similar to "followers") simonw 9599 closed 0     2 2019-11-09T20:20:20Z 2022-09-20T05:05:03Z 2020-02-07T07:03:28Z MEMBER  

Current list of commands: followers Save followers for specified user (defaults to... followers-ids Populate followers table with IDs of account followers friends-ids Populate followers table with IDs of account friends Obvious omission here is friends, which would be powered by https://api.twitter.com/1.1/friends/list.json: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friends-list

 twitter-to-sqlite 206156866 issue     
{
    "url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1378640768 I_kwDOBm6k_c5SLGOA 1816 Validate settings.json on startup in configuration directory mode simonw 9599 closed 0     2 2022-09-19T23:35:18Z 2022-09-20T01:15:48Z 2022-09-20T01:15:48Z OWNER  

It might have been useful for Datasette to show an error when started against a settings.json file that contains an invalid setting though.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1814#issuecomment-1251677554

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1816/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1353196970 I_kwDOCGYnMM5QqCWq 476 Release notes for 3.29 simonw 9599 closed 0   3.29 8355157 2 2022-08-27T23:21:21Z 2022-08-28T04:07:15Z 2022-08-28T04:07:03Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/476/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1352946135 I_kwDOCGYnMM5QpFHX 472 Reuse the locals/globals fix from --functions for other code accepting options simonw 9599 closed 0   3.29 8355157 2 2022-08-27T05:12:05Z 2022-08-27T05:20:12Z 2022-08-27T05:20:12Z OWNER  

I figured out a workaround for the ugly global x hack here: - https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229120653

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/472/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1319881016 PR_kwDOCGYnMM48Mmde 457 Link to installation instructions simonw 9599 closed 0   3.29 8355157 2 2022-07-27T17:38:36Z 2022-08-27T03:55:52Z 2022-07-27T17:57:50Z OWNER simonw/sqlite-utils/pulls/457 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/457/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1342357149 PR_kwDOCGYnMM49Wsnq 465 beanbag-docutils>=2.0 simonw 9599 closed 0     2 2022-08-17T22:41:39Z 2022-08-17T23:38:07Z 2022-08-17T23:38:02Z OWNER simonw/sqlite-utils/pulls/465

Refs #464

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/465/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1339444565 I_kwDOBm6k_c5P1k1V 1783 Better guidance as to what to do after you've installed Datasette simonw 9599 open 0     2 2022-08-15T20:11:06Z 2022-08-15T20:14:01Z   OWNER  

Feedback from Discord:

hello, love the project and came for help and to point out a possible gap in the docs. starting with "getting started" and "installation" every thing looks great, but then there's a giant leap after you have it installed and running. from the user perspective of "i have a csv of set of csvs that i want to turn into a table(s), what do i do next?" --- so something like maybe a page for creating your first project should go after "installation".

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1783/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1338278056 I_kwDOBm6k_c5PxICo 1782 Release notes for Datasette 0.62 simonw 9599 closed 0   Datasette 0.62 8303187 2 2022-08-14T15:26:45Z 2022-08-14T17:40:45Z 2022-08-14T17:32:54Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1782/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1223527226 I_kwDOBm6k_c5I7Ys6 1738 "Cannot use _sort and _sort_desc at the same time" simonw 9599 closed 0   Datasette 0.62 8303187 2 2022-05-03T01:06:24Z 2022-08-14T16:13:55Z 2022-08-14T16:13:55Z OWNER  

Triggered this error while playing with the sort desc checkbox and the apply button that are only visible on this page at mobile screen width:

https://latest.datasette.io/fixtures/compound_three_primary_keys?_sort_desc=pk1

Navigate to that page (with the browser narrow enough to show the box), un-check the box and click Apply:

Also notable: I managed to get to a page with ?_sort_desk=pk1 in the URL three times by clicking around with that button.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1738/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1326391841 PR_kwDOCGYnMM48iLGF 462 Discord badge simonw 9599 closed 0     2 2022-08-02T20:56:04Z 2022-08-02T21:15:57Z 2022-08-02T21:15:52Z OWNER simonw/sqlite-utils/pulls/462

Also testing fix for: - https://github.com/readthedocs/readthedocs-preview/issues/10

:books: Documentation preview :books:: https://sqlite-utils--462.org.readthedocs.build/en/462/

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/462/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1298531653 I_kwDOCGYnMM5NZgVF 451 Make sqlite_utils.utils.chunks a documented function simonw 9599 closed 0     2 2022-07-08T06:01:04Z 2022-07-15T22:09:34Z 2022-07-15T21:59:33Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/451/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1277328147 I_kwDOCGYnMM5MInsT 446 Use Just to automate running tests and linters locally simonw 9599 closed 0     2 2022-06-20T19:51:09Z 2022-06-21T19:28:35Z 2022-06-20T19:54:50Z OWNER  

I keep committing code that fails additional tests like mypy and flake8 and black. Automate those using Just.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/446/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1269998342 I_kwDOCGYnMM5LsqMG 443 Make `utils.rows_from_file()` a documented API simonw 9599 closed 0     2 2022-06-13T21:53:24Z 2022-06-20T19:49:37Z 2022-06-14T20:12:46Z OWNER  

rows_from_file() isn't part of the documented API but maybe it should be!

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1154385916

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/443/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1269886084 I_kwDOCGYnMM5LsOyE 442 `maximize_csv_field_size_limit()` utility function simonw 9599 closed 0     2 2022-06-13T19:54:54Z 2022-06-14T21:55:15Z 2022-06-14T21:31:49Z OWNER  

This code here runs only if cli.py is imported: https://github.com/simonw/sqlite-utils/blob/7ddf5300886a32d6daf60cf1d71efe492b65c87e/sqlite_utils/cli.py#L50-L59

I found myself needing the same fix in another library:

It should be a documented utility function.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/442/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1243704847 I_kwDOCGYnMM5KIW4P 435 Switch to Furo documentation theme simonw 9599 closed 0     2 2022-05-20T21:46:39Z 2022-05-20T21:56:10Z 2022-05-20T21:54:43Z OWNER   sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/435/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1243517592 I_kwDOBm6k_c5KHpKY 1748 Add copy buttons next to code examples in the documentation simonw 9599 closed 0     2 2022-05-20T19:09:00Z 2022-05-20T19:15:00Z 2022-05-20T19:11:32Z OWNER  

Similar to the ones in datasette-copyable which are implemented here: https://github.com/executablebooks/sphinx-copybutton/tree/f84c001a0507f8ec46779d0701b079a265564583

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1748/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1223177069 PR_kwDOCGYnMM43LrKB 429 Depend on click-default-group-wheel simonw 9599 closed 0     2 2022-05-02T18:03:10Z 2022-05-02T18:52:42Z 2022-05-02T18:05:00Z OWNER simonw/sqlite-utils/pulls/429

Trying to get this to work with Pyodide.

Refs: https://github.com/simonw/click-default-group-wheel/issues/3

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/429/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
1185868354 I_kwDOBm6k_c5GrupC 1695 Option to un-filter facet not shown for `?col__exact=value` simonw 9599 open 0     2 2022-03-30T04:44:02Z 2022-03-30T04:46:18Z   OWNER  

Spotted this on a page with COUNTY__exact=Lee in the URL:

With COUNTY=Lee you get this instead:

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1695/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1182141761 I_kwDOBm6k_c5Gdg1B 1690 Idea: `datasette.set_actor_cookie(response, actor)` simonw 9599 open 0     2 2022-03-26T22:41:52Z 2022-03-26T22:43:00Z   OWNER  

I just wrote this code in a plugin and it felt like it could benefit from an abstraction: https://github.com/simonw/datasette-auth0/blob/152e6eb21e96e9b73bd9c205f9749a1297d0ef0b/datasette_auth0/init.py#L79-L92

python redirect_response = Response.redirect("/") expires_at = int(time.time()) + (24 * 60 * 60) redirect_response.set_cookie( "ds_actor", datasette.sign( { "a": profile_response.json(), "e": baseconv.base62.encode(expires_at), }, "actor", ), ) return redirect_response

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1690/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1177101697 I_kwDOBm6k_c5GKSWB 1681 Potential bug in numeric handling where_clause for filters simonw 9599 open 0     2 2022-03-22T17:43:50Z 2022-03-22T17:49:09Z   OWNER  

Note that Datasette does already have special logic to convert parameters to integers for numeric comparisons like >:

https://github.com/simonw/datasette/blob/c4c9dbd0386e46d2bf199f0ed34e4895c98cb78c/datasette/filters.py#L203-L212

Though... it looks like there's a bug in that? It doesn't account for float values - "3.5".isdigit() return False - probably for the best, because int(3.5) would break that value anyway.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1671#issuecomment-1075432283

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1681/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
910088936 MDU6SXNzdWU5MTAwODg5MzY= 1355 datasette --get should efficiently handle streaming CSV simonw 9599 open 0     2 2021-06-03T04:40:40Z 2022-03-20T22:38:53Z   OWNER  

It would be great if you could use datasette --get to run queries that return streaming CSV data without running out of RAM.

Current implementation looks like it loads the entire result into memory first: https://github.com/simonw/datasette/blob/f78ebdc04537a6102316d6dbbf6c887565806078/datasette/cli.py#L546-L552

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1355/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1174404647 I_kwDOBm6k_c5F__4n 1669 Release 0.61 alpha simonw 9599 closed 0     2 2022-03-20T00:35:35Z 2022-03-20T01:24:36Z 2022-03-20T01:24:36Z OWNER  

I'm going to release this as a 0.61 alpha so I can more easily depend on it from datasette-hashed-urls.

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1669/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1170554975 I_kwDOBm6k_c5FxUBf 1663 Document the internals that were used in datasette-hashed-urls simonw 9599 closed 0   Datasette 1.0 3268330 2 2022-03-16T05:17:08Z 2022-03-19T04:04:50Z 2022-03-17T21:32:38Z OWNER  

The https://github.com/simonw/datasette-hashed-urls used a couple of currently undocumented features: - db.hash - Datasette(..., immutables=[...])

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1663/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1122416919 I_kwDOBm6k_c5C5rkX 1623 /-/patterns returns link: alternate JSON header to 404 simonw 9599 closed 0   Datasette 1.0 3268330 2 2022-02-02T21:42:49Z 2022-03-19T04:04:49Z 2022-02-02T21:48:56Z OWNER  

Bug from: - #1620

% curl -s -I 'https://latest.datasette.io/-/patterns' | grep link link: https://latest.datasette.io/-/patterns.json; rel="alternate"; type="application/json+datasette"

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1623/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
626593402 MDU6SXNzdWU2MjY1OTM0MDI= 780 Internals documentation for datasette.metadata() method simonw 9599 open 0   Datasette 1.0 3268330 2 2020-05-28T15:14:22Z 2022-03-15T20:50:34Z   OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/780/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1161937073 I_kwDOBm6k_c5FQcCx 1653 Mechanism to default a table to sorting by multiple columns simonw 9599 open 0     2 2022-03-07T21:20:11Z 2022-03-07T21:23:39Z   OWNER  

Discussed in https://github.com/simonw/datasette/discussions/1652

<sup>Originally posted by **zaneselvans** March 7, 2022</sup> It's easy to tell datasette to sort tables using a single column, as [described in the docs](https://docs.datasette.io/en/stable/metadata.html#setting-a-default-sort-order): ```yaml databases: ferc1: tables: f1_edcfu_epda: sort: created_time ``` But is there some way to tell it to sort using a composite key, like you would in an `ORDER BY` clause instead? For example, the way it's being done **[in this query](https://data.catalyst.coop/ferc1?sql=select%0D%0A++rowid%2C%0D%0A++respondent_id%2C%0D%0A++report_year%2C%0D%0A++spplmnt_num%2C%0D%0A++row_number%2C%0D%0A++row_seq%2C%0D%0A++row_prvlg%2C%0D%0A++acct_num%2C%0D%0A++depr_plnt_base%2C%0D%0A++est_avg_srvce_lf%2C%0D%0A++net_salvage%2C%0D%0A++apply_depr_rate%2C%0D%0A++mrtlty_crv_typ%2C%0D%0A++avg_remaining_lf%2C%0D%0A++report_prd%0D%0Afrom%0D%0A++f1_edcfu_epda%0D%0Awhere%0D%0A++respondent_id+%3D+210%0D%0A++AND+report_year+%3D+2020%0D%0Aorder+by%0D%0A++report_year%2C+report_prd%2C+respondent_id%2C+spplmnt_num%2C+row_number%0D%0Alimit%0D%0A++1000)** on our Datasette? ```sql SELECT respondent_id, report_year, spplmnt_num, row_number, row_seq, row_prvlg, acct_num, depr_plnt_base, est_avg_srvce_lf, net_salvage, apply_depr_rate, mrtlty_crv_typ, avg_remaining_lf, report_prd FROM f1_edcfu_epda WHERE respondent_id = 210 AND report_year = 2020 ORDER BY report_year, report_prd, respondent_id, spplmnt_num, row_number LIMIT 1000 ``` The problem here is that by default it's using `rowid` (the SQLite assigned autoincrementing integer key) to order the records, but the table **should** have a natural composite primary key, but the original database that this data is being migrated from doesn't enforce unique primary keys, so there are dupes, and we don't want to drop those rows, and the records are somehow getting jumbled in the database (the `rowid` ordering isn't lined up with the expected ordering based on the composite primary key, though it's close) and this jumbling is confusing to users that expect to see the data ordered based on the natural primary key. I've tried setting the `sort` metadata parameter to a list of column names, a tuple of column names, a quoted string of comma-separated column names, a quoted string of a tuple of column names... ```yaml databases: ferc1: tables: f1_edcfu_epda: sort: "(report_year, report_prd, respondent_id, spplmnt_num, row_number)" ``` and they all give me server errors like: ``` Cannot sort table by (report_year, report_prd, respondent_id, spplmnt_num, row_number) ```
 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1653/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1128139375 I_kwDOCGYnMM5DPgpv 405 `Database(memory_name="name")` constructor argument simonw 9599 closed 0     2 2022-02-09T07:15:03Z 2022-02-16T01:23:16Z 2022-02-16T01:23:16Z OWNER  

SQLite in-memory databases can be named, in which case multiple connections can be opened to a shared in-memory database running within the same process.

Datasette supports this - SQLite could support it too.

https://docs.datasette.io/en/0.60.2/internals.html#database-ds-path-none-is-mutable-false-is-memory-false-memory-name-none

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/405/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1128120451 I_kwDOCGYnMM5DPcCD 404 Add example of `--convert` to the help for `sqlite-utils insert` simonw 9599 closed 0     2 2022-02-09T06:49:09Z 2022-02-09T06:56:35Z 2022-02-09T06:55:16Z OWNER  

https://sqlite-utils.datasette.io/en/3.23/cli-reference.html#insert would be more useful if it included an example of --convert in action.

I can maybe use an example from https://simonwillison.net/2022/Jan/11/sqlite-utils/

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/404/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1109783030 I_kwDOBm6k_c5CJfH2 1607 More detailed information about installed SpatiaLite version simonw 9599 closed 0   Datasette 1.0 3268330 2 2022-01-20T21:28:03Z 2022-02-09T06:42:02Z 2022-02-09T06:32:28Z OWNER  

https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.0.0.html#version has a whole bunch of interesting functions for things like freexl_version() and geos_version() and HasMathSQL() and suchlike.

These could be shown on the /-/versions page.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1607/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1125081640 I_kwDOCGYnMM5DD2Io 401 Update SpatiaLite example in the documentation simonw 9599 closed 0     2 2022-02-06T02:02:07Z 2022-02-06T02:05:03Z 2022-02-06T02:03:24Z OWNER  

This one here: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions

It should take advantage of the new methods from: - #79

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/401/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
723460107 MDU6SXNzdWU3MjM0NjAxMDc= 187 Maybe: Utility method / CLI tool for initializing SpatiaLite simonw 9599 closed 0     2 2020-10-16T19:04:03Z 2022-02-05T00:04:26Z 2020-10-16T19:15:13Z OWNER  

I think this should initialize SpatiaLite against the current database if it has not been initialized already.

Relevant code: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L112-L126

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/187/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
557825032 MDU6SXNzdWU1NTc4MjUwMzI= 77 Ability to insert data that is transformed by a SQL function simonw 9599 closed 0     2 2020-01-30T23:45:55Z 2022-02-05T00:04:25Z 2020-01-31T00:24:32Z OWNER  

I want to be able to run the equivalent of this SQL insert: ```python

Convert to "Well Known Text" format

wkt = shape(geojson['geometry']).wkt

Insert and commit the record

conn.execute("INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))", ( "Wales", wkt )) conn.commit() ``` From the Datasette SpatiaLite docs: https://datasette.readthedocs.io/en/stable/spatialite.html

To do this, I need a way of telling sqlite-utils that a specific column should be wrapped in GeomFromText(?, 4326).

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/77/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1114638930 I_kwDOCGYnMM5CcApS 391 `sqlite-utils bulk` progress bar simonw 9599 closed 0     2 2022-01-26T05:14:49Z 2022-01-26T05:17:20Z 2022-01-26T05:16:51Z OWNER  

It can easily have a progress bar because it works by looping through an iterator: https://github.com/simonw/sqlite-utils/blob/a9fca7efa4184fbb2a65ca1275c326950ed9d3c1/sqlite_utils/cli.py#L1014-L1018

Should also support the --silent option if I add this.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/391/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
471818939 MDU6SXNzdWU0NzE4MTg5Mzk= 48 Jupyter notebook demo of the library, launchable on Binder simonw 9599 closed 0     2 2019-07-23T17:05:05Z 2022-01-26T02:08:46Z 2022-01-26T02:08:39Z OWNER  
sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/48/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1114544727 I_kwDOCGYnMM5CbppX 389 Plausible analytics for documentation simonw 9599 closed 0     2 2022-01-26T01:58:35Z 2022-01-26T02:07:41Z 2022-01-26T02:07:41Z OWNER  

```html

<script defer data-domain="sqlite-utils.datasette.io" src="https://plausible.io/js/plausible.js"></script>

```

Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/388#issuecomment-1021785268

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/389/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
842416110 MDU6SXNzdWU4NDI0MTYxMTA= 1278 SpatiaLite timezones demo is broken simonw 9599 closed 0     2 2021-03-27T04:45:27Z 2022-01-20T21:29:43Z 2021-03-27T16:17:13Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1278/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
443040665 MDU6SXNzdWU0NDMwNDA2NjU= 466 Move "no such module: VirtualSpatialIndex" code elsewhere simonw 9599 closed 0   0.28 4305096 2 2019-05-11T22:09:00Z 2022-01-20T21:29:41Z 2019-05-11T22:57:22Z OWNER  

We currently show a useful warning (from #331) when the user tries to open a spatialite database without first loading the module:

https://github.com/simonw/datasette/blob/c692cd291111050483a32bea1ee08e994a0b781b/datasette/app.py#L547-L554

This code is part of .inspect() which is going away - see #462 - so I need to find somewhere else for it to live.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/466/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
563347679 MDU6SXNzdWU1NjMzNDc2Nzk= 668 Make it easier to load SpatiaLite simonw 9599 closed 0     2 2020-02-11T17:03:43Z 2022-01-20T21:29:41Z 2021-01-04T20:18:39Z OWNER  

``` $ datasette spatial.db Serve! files=('spatial.db',) (immutables=()) on port 8001 ERROR: conn=<sqlite3.Connection object at 0x11e388f10>, sql = 'PRAGMA table_info(SpatialIndex);', params = None: no such module: VirtualSpatialIndex Usage: datasette serve [OPTIONS] [FILES]...

Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module.

Read more: https://datasette.readthedocs.io/en/latest/spatialite.html This error message could sniff around in the common locations for the SpatiaLite module and output the CLI command you should use to enable it: datasette spatial.db --load-extension=/usr/local/lib/mod_spatialite.dylib `` Even better: if Datasette had a--spatialite` option which automatically loads the extension from common locations, if it can find it.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/668/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1097040427 I_kwDOBm6k_c5BY4Ir 1587 Add `sqlite_stat1`(-4) tables to hidden table list simonw 9599 closed 0     2 2022-01-08T21:28:20Z 2022-01-20T04:12:59Z 2022-01-20T04:12:59Z OWNER  

Running ANALYZE creates a new visible table called sqlite_stat1: https://www.sqlite.org/fileformat.html#the_sqlite_stat1_table

This should be added to the default list of hidden tables in Datasette.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1587/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
991467558 MDU6SXNzdWU5OTE0Njc1NTg= 1466 Add Datasette Desktop to installation documentation simonw 9599 closed 0   Datasette 0.60 7571612 2 2021-09-08T19:41:27Z 2022-01-13T22:28:28Z 2022-01-13T21:55:18Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1466/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083921371 I_kwDOBm6k_c5Am1Pb 1570 Separate db.execute_write() into three methods simonw 9599 closed 0   Datasette 0.60 7571612 2 2021-12-18T18:45:54Z 2022-01-13T22:27:38Z 2021-12-18T18:57:25Z OWNER  

Rather than adding a executemany=True parameter, I'm now thinking a better design might be to have three methods:

  • db.execute_write(sql, params=None, block=False)
  • db.execute_write_script(sql, block=False)
  • db.execute_write_many(sql, params_seq, block=False)

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997267416

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1570/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083895395 I_kwDOBm6k_c5Amu5j 1569 db.execute_write(..., executescript=True) parameter simonw 9599 closed 0   Datasette 0.60 7571612 2 2021-12-18T18:20:47Z 2022-01-13T22:27:27Z 2021-12-18T18:34:18Z OWNER  

Idea: teach execute_write to accept an optional executescript=True parameter, like this: ```diff diff --git a/datasette/database.py b/datasette/database.py index 468e936..1a424f5 100644 --- a/datasette/database.py +++ b/datasette/database.py @@ -94,10 +94,14 @@ class Database: f"file:{self.path}{qs}", uri=True, check_same_thread=False )

  • async def execute_write(self, sql, params=None, block=False):
  • async def execute_write(self, sql, params=None, executescript=False, block=False):
  • assert not executescript and params, "Cannot use params with executescript=True" def _inner(conn): with conn:
  • return conn.execute(sql, params or [])
  • if executescript:
  • return conn.executescript(sql)
  • else:
  • return conn.execute(sql, params or [])
     with trace("sql", database=self.name, sql=sql.strip(), params=params):
     results = await self.execute_write_fn(_inner, block=block)

    ```

Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997248364

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1569/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083726550 I_kwDOBm6k_c5AmFrW 1568 Trace should show queries on the write connection too simonw 9599 closed 0   Datasette 0.60 7571612 2 2021-12-18T02:34:12Z 2022-01-13T22:27:23Z 2021-12-18T02:42:34Z OWNER   datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1568/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1083573206 I_kwDOBm6k_c5AlgPW 1563 Datasette(... files=) should not be a required argument simonw 9599 closed 0   Datasette 0.60 7571612 2 2021-12-17T19:54:18Z 2022-01-13T22:27:18Z 2021-12-18T02:19:40Z OWNER  

```pycon

ds = Datasette(memory=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: init() missing 1 required positional argument: 'files' ds = Datasette(memory=True, files=[]) `` I wanted to create an in-memory Datasette for running some tests, no point in forcing me to passfiles=[]` to do that.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1563/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
712202333 MDU6SXNzdWU3MTIyMDIzMzM= 982 SQL editor should allow execution of write queries, if you have permission simonw 9599 open 0     2 2020-09-30T19:04:35Z 2022-01-13T22:21:29Z   OWNER  

The datasette-write plugin provides this at the moment https://github.com/simonw/datasette-write - but it feels like it should be a built-in capability, protected by a default permission.

UI concept: if you have write permission then the existing SQL editor gets an "execute write" checkbox underneath it.

JavaScript can spot if you appear to be trying to execute an UPDATE or INSERT or DELETE query and check that checkbox for you.

If you link to a query page with a non-SELECT then that query will be displayed in the box ready for you to POST submit it. The page will also then get "cannot be embedded" headers to protect against clickjacking.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/982/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    
1099584685 I_kwDOCGYnMM5BilSt 381 `sqlite-utils rows` options `--limit` and `--offset` simonw 9599 closed 0     2 2022-01-11T20:23:12Z 2022-01-11T23:33:37Z 2022-01-11T23:19:36Z OWNER  

Because I often want to use it just to preview a few rows from the database. Piping through | head -n 20 works for JSON and CSV (they stream) but not for --table.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/381/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1097041471 PR_kwDOCGYnMM4wsVM6 367 Initial prototype of .analyze() methods simonw 9599 closed 0   3.21 7558727 2 2022-01-08T21:35:12Z 2022-01-10T19:31:08Z 2022-01-10T19:31:08Z OWNER simonw/sqlite-utils/pulls/367

Refs #366

 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/367/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 1  
1077628073 I_kwDOBm6k_c5AO0yp 1550 Research option for returning all rows from arbitrary query simonw 9599 open 0     2 2021-12-11T19:31:11Z 2021-12-11T23:43:24Z   OWNER  

Inspired by thinking about #1549 - returning ALL rows from an arbitrary query is a lot easier if you just run that query and keep iterating over the cursor.

I've avoided doing that in the past because it could tie up a connection for a long time - but in private instances this wouldn't be such a problem.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1550/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    

Next page

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);