id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,pull_request,body,repo,type,active_lock_reason,performed_via_github_app,reactions,draft,state_reason
2029161033,I_kwDOCGYnMM548opJ,606,str and int as aliases for text and integer,9599,closed,0,,,2,2023-12-06T18:35:49Z,2023-12-06T19:44:04Z,2023-12-06T18:49:32Z,OWNER,,"I keep making this mistake:
```bash
sqlite-utils add-column content.db assets _since int
```
```
Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME [[integer|float|b
lob|text|INTEGER|FLOAT|BLOB|TEXT]]
Try 'sqlite-utils add-column -h' for help.
Error: Invalid value for '[[integer|float|blob|text|INTEGER|FLOAT|BLOB|TEXT]]':
'int' is not one of 'integer', 'float', 'blob', 'text', 'INTEGER', 'FLOAT', 'BLOB', 'TEXT'.
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/606/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1910269679,I_kwDOBm6k_c5x3Gbv,2196,Discord invite link returns 401,1892194,closed,0,,,2,2023-09-24T15:16:54Z,2023-10-13T00:07:08Z,2023-10-12T21:54:54Z,NONE,,"I found the link to the datasette discord channel via [this query](https://github.com/search?q=repo%3Asimonw%2Fdatasette%20discord&type=code).
The following video should be self explanatory:
https://github.com/simonw/datasette/assets/1892194/8cd33e88-bcaa-41f3-9818-ab4d589c3f02
Link for reference: https://discord.com/invite/ktd74dm5mw",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2196/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1846076261,I_kwDOBm6k_c5uCONl,2139,border-color: ##ff0000 bug - two hashes,9599,closed,0,,8755003,2,2023-08-11T01:22:58Z,2023-08-11T05:16:24Z,2023-08-11T05:16:24Z,OWNER,,"Spotted this on https://latest.datasette.io/extra_database
```html
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2139/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1844213115,I_kwDOBm6k_c5t7HV7,2138,on_success_message_sql option for writable canned queries,9599,closed,0,,8755003,2,2023-08-10T00:20:14Z,2023-08-10T00:39:40Z,2023-08-10T00:34:26Z,OWNER,,"> Or... how about if the `on_success_message` option could define a SQL query to be executed to generate that message? Maybe `on_success_message_sql`.
- https://github.com/simonw/datasette/issues/2134",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2138/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1840329615,I_kwDOBm6k_c5tsTOP,2130,Render plugin mechanism needs `error` and `truncated` fields,9599,closed,0,,9700784,2,2023-08-07T23:19:19Z,2023-08-08T01:51:54Z,2023-08-08T01:47:42Z,OWNER,,"While working on:
- https://github.com/simonw/datasette/pull/2118
It became clear that the `render` callback function documented here: https://docs.datasette.io/en/0.64.3/plugin_hooks.html#register-output-renderer-datasette
Needs to grow the ability to be told if an error occurred (an `error` string) and if the results were truncated (a `truncated` boolean).",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2130/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816997390,I_kwDOCGYnMM5sTS4O,576,Backfill the release notes prior to 0.4,9599,closed,0,,,2,2023-07-23T05:41:42Z,2023-07-23T05:49:51Z,2023-07-23T05:48:21Z,OWNER,,"Currently the changelog starts at 0.4:
https://sqlite-utils.datasette.io/en/3.34/changelog.html#id115
I want the other releases - according to https://pypi.org/project/sqlite-utils/#history there are three missing:
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/576/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816919568,I_kwDOCGYnMM5sS_4Q,575,Python API ability to opt-out of connection plugins,9599,closed,0,,,2,2023-07-22T23:01:13Z,2023-07-22T23:17:22Z,2023-07-22T23:08:22Z,OWNER,,"Plugins affecting the CLI by default makes sense to me.
I'm less confident about them _always_ affecting users of the Python API.
I'm going to have them apply by default, but I'm going to add a mechanism to opt-out on an individual database basis. Basically this:
```python
from sqlite_utils import Database
db = Database(memory=True, execute_plugins=False)
# Anything using db from here on will not execute plugins
```
cc @asg017
Refs:
- #567
- #574 ",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/575/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1816877910,I_kwDOCGYnMM5sS1tW,572,Don't test Python 3.7 against textual,9599,closed,0,,,2,2023-07-22T19:57:03Z,2023-07-22T22:16:50Z,2023-07-22T22:16:50Z,OWNER,,"Spotted this in the GitHub Actions logs:
![IMG_5046](https://github.com/simonw/sqlite-utils/assets/9599/81fb1093-cd8a-4019-a612-2e49b500c933)
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/572/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1795187493,I_kwDODLZ_YM5rAGMl,12,Switch to pyproject.toml,9599,closed,0,,,2,2023-07-09T01:06:56Z,2023-07-09T01:19:43Z,2023-07-09T01:19:42Z,MEMBER,,First of my CLI tools to use https://til.simonwillison.net/python/pyproject,213286752,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1781047747,I_kwDOBm6k_c5qKKHD,2092,test_homepage intermittent failure,9599,closed,0,,,2,2023-06-29T15:20:37Z,2023-06-29T15:26:28Z,2023-06-29T15:24:13Z,OWNER,,"e.g. in https://github.com/simonw/datasette/actions/runs/5413590227/jobs/9839373852
```
=================================== FAILURES ===================================
________________________________ test_homepage _________________________________
[gw0] linux -- Python 3.7.17 /opt/hostedtoolcache/Python/3.7.17/x64/bin/python
ds_client =
@pytest.mark.asyncio
async def test_homepage(ds_client):
response = await ds_client.get(""/.json"")
assert response.status_code == 200
assert ""application/json; charset=utf-8"" == response.headers[""content-type""]
data = response.json()
assert data.keys() == {""fixtures"": 0}.keys()
d = data[""fixtures""]
assert d[""name""] == ""fixtures""
assert d[""tables_count""] == 24
assert len(d[""tables_and_views_truncated""]) == 5
assert d[""tables_and_views_more""] is True
# 4 hidden FTS tables + no_primary_key (hidden in metadata)
assert d[""hidden_tables_count""] == 6
# 201 in no_primary_key, plus 6 in other hidden tables:
> assert d[""hidden_table_rows_sum""] == 207, data
E AssertionError: {'fixtures': {'color': '9403e5', 'hash': None, 'hidden_table_rows_sum': 0, 'hidden_tables_count': 6, ...}}
E assert 0 == 207
```
My guess is that this is a timing error, where very occasionally the ""count rows but stop counting if it exceeds a time limit"" thing fails.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2092/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1655860104,I_kwDOCGYnMM5ismuI,535,rows: --transpose or psql extended view-like functionality,7908073,closed,0,,,2,2023-04-05T15:37:33Z,2023-06-15T08:39:49Z,2023-06-14T22:05:28Z,CONTRIBUTOR,,"It would be nice if the rows subcommand had a flag, perhaps called `--transpose` which would print in long form instead of wide. Similar to extended display mode in psql (`\x`)
In other words instead of this:
```
sqlite-utils rows --limit 5 --fmt github track_metadata.db songs
```
| track_id | title | song_id | release | artist_id | artist_mbid | artist_name | duration | artist_familiarity | artist_hotttnesss | year | track_7digitalid | shs_perf | shs_work |
|--------------------|-------------------|--------------------|--------------------------------------|--------------------|--------------------------------------|------------------|------------|----------------------|---------------------|--------|--------------------|------------|------------|
| TRMMMYQ128F932D901 | Silent Night | SOQMMHC12AB0180CB8 | Monster Ballads X-Mas | ARYZTJS1187B98C555 | 357ff05d-848a-44cf-b608-cb34b5701ae5 | Faster Pussy cat | 252.055 | 0.649822 | 0.394032 | 2003 | 7032331 | -1 | 0 |
| TRMMMKD128F425225D | Tanssi vaan | SOVFVAK12A8C1350D9 | Karkuteillä | ARMVN3U1187FB3A1EB | 8d7ef530-a6fd-4f8f-b2e2-74aec765e0f9 | Karkkiautomaatti | 156.551 | 0.439604 | 0.356992 | 1995 | 1514808 | -1 | 0 |
| TRMMMRX128F93187D9 | No One Could Ever | SOGTUKN12AB017F4F1 | Butter | ARGEKB01187FB50750 | 3d403d44-36ce-465c-ad43-ae877e65adc4 | Hudson Mohawke | 138.971 | 0.643681 | 0.437504 | 2006 | 6945353 | -1 | 0 |
| TRMMMCH128F425532C | Si Vos Querés | SOBNYVR12A8C13558C | De Culo | ARNWYLR1187B9B2F9C | 12be7648-7094-495f-90e6-df4189d68615 | Yerba Brava | 145.058 | 0.448501 | 0.372349 | 2003 | 2168257 | -1 | 0 |
| TRMMMWA128F426B589 | Tangle Of Aspens | SOHSBXH12A8C13B0DF | Rene Ablaze Presents Winter Sessions | AREQDTE1269FB37231 | | Der Mystic | 514.298 | 0 | 0 | 0 | 2264873 | -1 | 0 |
The output would look something like this:
```
$ for col in (sqlite-columns track_metadata.db songs)
sqlite-utils --fmt github track_metadata.db ""select $col from songs order by rowid desc limit 5""
end
```
| track_id |
|--------------------|
| TRYYYVU12903CD01E3 |
| TRYYYDJ128F9310A21 |
| TRYYYMG128F4260ECA |
| TRYYYJO128F426DA37 |
| TRYYYUS12903CD2DF0 |
| title |
|-------------------------------------|
| Fernweh feat. Sektion Kuchikäschtli |
| Faraday |
| Novemba |
| Jago Chhadeo |
| O Samba Da Vida |
| song_id |
|--------------------|
| SOWXJXQ12AB0189F43 |
| SOLXGOR12A81C21EB7 |
| SOHODZI12A8C137BB3 |
| SOXQYIQ12A8C137FBB |
| SOTXAME12AB018F136 |
| release |
|---------------------------------|
| So Oder So |
| The Trance Collection Vol. 2 |
| Dub_Connected: electronic music |
| Naale Baba Lassi Pee Gya |
| Pacha V.I.P. |
| artist_id |
|--------------------|
| AR7PLM21187B990D08 |
| ARCMCOK1187B9B1073 |
| ARZ3R6M1187B9AF750 |
| ART5FZD1187B9A7FCF |
| AR7Z4J81187FB3FC59 |
| artist_mbid |
|--------------------------------------|
| 3af2b07e-c91c-4160-9bda-f0b9e3144ed3 |
| 4ac5f3de-c5ad-475e-ad50-41f1ef9dba20 |
| 8b97e9c8-61f5-4615-9a96-276f24204e34 |
| 2357c400-9109-42b6-b3fe-9e2d9f8e3872 |
| 9d50cb20-7e42-45cc-b0dd-154c3e92a577 |
| artist_name |
|----------------|
| Texta |
| Elude |
| Gabriel Le Mar |
| Kuldeep Manak |
| Kiko Navarro |
| duration |
|------------|
| 295.079 |
| 484.519 |
| 553.038 |
| 244.166 |
| 217.443 |
| artist_familiarity |
|----------------------|
| 0.552977 |
| 0.403668 |
| 0.556918 |
| 0.4015 |
| 0.528617 |
| artist_hotttnesss |
|---------------------|
| 0.454869 |
| 0.256935 |
| 0.336914 |
| 0.374866 |
| 0.411595 |
| year |
|--------|
| 2004 |
| 0 |
| 0 |
| 0 |
| 0 |
| track_7digitalid |
|--------------------|
| 8486723 |
| 5472456 |
| 2219291 |
| 1632096 |
| 7522478 |
| shs_perf |
|------------|
| -1 |
| -1 |
| -1 |
| -1 |
| -1 |
| shs_work |
|------------|
| 0 |
| 0 |
| 0 |
| 0 |
| 0 |
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/535/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1740150327,I_kwDOCGYnMM5nuJY3,557,Aliased ROWID option for tables created from alter=True commands,7908073,closed,0,,,2,2023-06-04T05:29:28Z,2023-06-14T06:09:21Z,2023-06-05T19:26:26Z,CONTRIBUTOR,,"> If you use INTEGER PRIMARY KEY column, the VACUUM does not change the values of that column. However, if you use unaliased rowid, the VACUUM command will reset the rowid values.
ROWID should never be used with foreign keys but the simple act of aliasing rowid to id (which is what happens when one does `id integer primary key` DDL) makes it OK.
It would be convenient if there were more options to use a string column (eg. filepath) as the PK, and be able to use it during upserts, but when creating a foreign key, to create an integer column which aliases rowid
I made an attempt to switch to integer primary keys here but it is not going well... In my usecase the path column is a business key. Yes, it should be as simple as including the `id` column in any select statement where I plan on using `upsert` but it would be nice if this could be abstracted away somehow https://github.com/chapmanjacobd/library/commit/788cd125be01d76f0fe2153335d9f6b21db1343c
https://github.com/chapmanjacobd/library/actions/runs/5173602136/jobs/9319024777",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/557/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1718515590,I_kwDOCGYnMM5mbneG,544,New options for analyze-tables --common-limit --no-most and --no-least,9599,closed,0,,,2,2023-05-21T14:03:19Z,2023-05-21T17:03:06Z,2023-05-21T16:19:31Z,OWNER,,"The ""least common"" section is frequently uninteresting, especially for huge tables with a large number of repeated-once values.
sqlite-utils analyze-tables content.db repos --common-limit 20 --no-least",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/544/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1702354223,I_kwDOBm6k_c5ld90v,2070,Mechanism for deploying a preview of a branch using Vercel,9599,closed,0,,,2,2023-05-09T16:21:45Z,2023-05-09T16:25:00Z,2023-05-09T16:24:31Z,OWNER,,"I prototyped that here: https://github.com/simonw/one-off-actions/blob/main/.github/workflows/deploy-datasette-branch-preview.yml
It deployed the `json-extras-query` branch here: https://datasette-preview-json-extras-query.vercel.app/",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2070/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1620254998,I_kwDOCGYnMM5gkyEW,532,Show more information when JSON can't be imported with sqlite-utils insert,83080728,closed,0,,,2,2023-03-12T06:41:44Z,2023-05-08T20:32:16Z,2023-05-08T20:32:02Z,NONE,,"I am currently trying to import the [JSON export of my data from Discord](https://support.discord.com/hc/en-us/articles/360004027692-Requesting-a-Copy-of-your-Data), specifically `activity/reporting/events-*.json`
```
sqlite-utils.exe insert test.db reporting events-2023-00000-of-00001.json
[###################################-] 99% 00:00:00
Error: Invalid JSON - use --csv for CSV or --tsv for TSV files
```
Please show more information as to *why* this is invalid, if possible.
I am using version 3.30 with Python 3.10 on Windows 11.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/532/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1633077183,I_kwDOBm6k_c5hVse_,2041,Remove obsolete table POST code,9599,closed,0,,8755003,2,2023-03-21T01:01:40Z,2023-03-21T01:17:44Z,2023-03-21T01:17:43Z,OWNER,,"Spotted this in:
- #1999
`POST /db/table` currently executes obsolete code for inserting a row - I replaced that with `/db/table/-/insert` in
https://github.com/simonw/datasette/commit/6e788b49edf4f842c0817f006eb9d865778eea5e but forgot to remove the old code.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2041/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1620516340,I_kwDOCGYnMM5glx30,533,ReadTheDocs error: not all arguments converted during string formatting,9599,closed,0,,,2,2023-03-12T21:21:05Z,2023-03-12T21:25:33Z,2023-03-12T21:25:33Z,OWNER,,"This came up as a failure running tests for:
- #531
Traceback on https://readthedocs.org/projects/sqlite-utils/builds/19749348/
```
File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/docutils/parsers/rst/states.py"", line 889, in interpreted
nodes, messages2 = role_fn(role, rawsource, text, lineno, self)
File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py"", line 103, in role
title = caption % part
TypeError: not all arguments converted during string formatting
Exception occurred:
File ""/home/docs/checkouts/readthedocs.org/user_builds/sqlite-utils/envs/531/lib/python3.8/site-packages/sphinx/ext/extlinks.py"", line 103, in role
title = caption % part
TypeError: not all arguments converted during string formatting
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/533/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1612296210,I_kwDOBm6k_c5gGbAS,2033,`datasette install -r requirements.txt`,9599,closed,0,,,2,2023-03-06T22:17:17Z,2023-03-06T22:54:52Z,2023-03-06T22:27:34Z,OWNER,,"Would be useful for cases where you want to install a whole set of plugins in one go, e.g. when running tutorials in GitHub Codespaces.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2033/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1594383280,I_kwDOBm6k_c5fCFuw,2030,How to use Datasette with apache webserver on GCP?,19700859,closed,0,,,2,2023-02-22T03:08:49Z,2023-02-22T21:54:39Z,2023-02-22T21:54:39Z,NONE,,"Hi Simon and Datasette team-
I have installed apache2 webserver inside GCP VM using apt.
I can see my ""Hello World"" index.html if I use the external IP of this GCP in a browser.
However, when I try to run datasette with different combinations of -h and -p, I am still unable to access the webpage.
I cannot invest Docker on this VM.
Any pointers to use datasette with already existing apache2 webserver on GCP is appreciated.
Thanks.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2030/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1579695809,I_kwDOBm6k_c5eKD7B,2023,Error: Invalid setting 'hash_urls' in settings.json in 0.64.1,80409402,closed,0,,,2,2023-02-10T13:35:01Z,2023-02-10T15:40:00Z,2023-02-10T15:39:59Z,NONE,,"On a Debian machine, using datasette 0.64.1 installed with `pip3`, I am getting a `datasette[114272]: Error: Invalid setting 'hash_urls' in settings.json` in `journalctl -xe`. The same settings work on 0.54.1 on another Debian server.
This is my `settings.json`:
```json
{
""default_page_size"": 200,
""max_returned_rows"": 8000,
""num_sql_threads"": 3,
""sql_time_limit_ms"": 1000,
""default_facet_size"": 30,
""facet_time_limit_ms"": 200,
""facet_suggest_time_limit_ms"": 50,
""hash_urls"": false,
""allow_facet"": true,
""allow_download"": true,
""suggest_facets"": true,
""default_cache_ttl"": 5,
""default_cache_ttl_hashed"": 31536000,
""cache_size_kb"": 0,
""allow_csv_stream"": true,
""max_csv_mb"": 100,
""truncate_cells_html"": 2048,
""force_https_urls"": false,
""template_debug"": false,
""base_url"": ""/pclim/db/""
}
```
This looks ok to me. Would you have any ideas?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/2023/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1515182998,I_kwDOBm6k_c5aT9uW,1970,"Path ""None"" in _internal database table",9599,closed,0,,,2,2022-12-31T18:51:05Z,2022-12-31T19:22:58Z,2022-12-31T18:52:49Z,OWNER,,"See https://latest.datasette.io/_internal/databases (after https://latest.datasette.io/login-as-root)
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1970/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1501713288,I_kwDOBm6k_c5ZglOI,1963,0.63.3 bugfix release,9599,closed,0,,,2,2022-12-18T02:48:15Z,2022-12-18T03:26:55Z,2022-12-18T03:26:55Z,OWNER,,"I'm going to ship a release which back-ports these two fixes:
- https://github.com/simonw/datasette/issues/1958
- https://github.com/simonw/datasette/issues/1955",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1963/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1495821607,I_kwDOBm6k_c5ZKG0n,1953,Release notes for Datasette 1.0a2,9599,closed,0,,8711695,2,2022-12-14T06:26:40Z,2022-12-15T02:02:15Z,2022-12-15T02:01:08Z,OWNER,,"https://github.com/simonw/datasette/milestone/27?closed=1
https://github.com/simonw/datasette/compare/1.0a1...9ad76d279e2c3874ca5070626a25458ce129f126",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1953/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1483320357,I_kwDOBm6k_c5Yaawl,1937,/db/-/create API should require insert-rows permission to use row: or rows: option,9599,closed,0,,8711695,2,2022-12-08T01:33:09Z,2022-12-14T20:21:26Z,2022-12-14T20:21:26Z,OWNER,,Otherwise someone with `create-table` but no` insert-rows` permission could abuse it to insert data.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1937/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1497288666,I_kwDOBm6k_c5ZPs_a,1956,Handle abbreviations properly in permission_allowed_actor_restrictions,9599,closed,0,,8711695,2,2022-12-14T19:54:21Z,2022-12-14T20:04:29Z,2022-12-14T20:04:28Z,OWNER,,"This code currently assumes abbreviations are:
```pyton
action_initials = """".join([word[0] for word in action.split(""-"")])
```
https://github.com/simonw/datasette/blob/1a3dcf494376e32f7cff110c86a88e5b0a3f3924/datasette/default_permissions.py#L182-L208
That's no longer correct, they are now registered by the new plugin hook:
- #1939 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1956/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1495241162,I_kwDOBm6k_c5ZH5HK,1950,"Bad ?_sort returns a 500 error, should be a 400",9599,closed,0,,,2,2022-12-13T22:08:16Z,2022-12-13T22:23:22Z,2022-12-13T22:23:22Z,OWNER,,"https://latest.datasette.io/fixtures/facetable?_sort=bad
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1950/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1493339206,I_kwDOBm6k_c5ZAoxG,1946,`datasette --get` mechanism for sending tokens,9599,closed,0,,8711695,2,2022-12-13T04:25:05Z,2022-12-13T04:36:57Z,2022-12-13T04:36:57Z,OWNER,,"> For the tests for `datasette create-token` it would be useful if `datasette --get` had a mechanism for sending an `Authorization: Bearer X` header.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1855#issuecomment-1347731288_
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1946/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1469043836,I_kwDOBm6k_c5Xj9R8,1917,Don't allow writable API to edit the `_memory` database,9599,closed,0,,7867486,2,2022-11-30T04:51:59Z,2022-11-30T05:07:56Z,2022-11-30T05:07:55Z,OWNER,,"It shows up on https://latest.datasette.io/-/api (once you are signed in as root) - but there's no point in creating tables in it because they likely won't persist from one request to the next, as it's not a shared named database.
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1917/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1468519699,I_kwDOBm6k_c5Xh9UT,1911,`/db/-/create` should support creating tables with compound primary keys,9599,closed,0,,8658075,2,2022-11-29T18:30:47Z,2022-11-29T18:50:58Z,2022-11-29T18:48:05Z,OWNER,,"Found myself needing this to write the tests for:
- #1864 ",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1911/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1447388809,I_kwDOBm6k_c5WRWaJ,1887,Add a confirm step to the drop table API,9599,closed,0,,8658075,2,2022-11-14T04:59:53Z,2022-11-15T19:59:59Z,2022-11-14T05:18:51Z,OWNER,,"> In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1871#issuecomment-1313097057_
Added drop table API in:
- #1874",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1887/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1436539554,I_kwDOCGYnMM5Vn9qi,511,"[insert_all, upsert_all] IntegrityError: constraint failed",7908073,closed,0,,,2,2022-11-04T19:21:48Z,2022-11-04T22:59:54Z,2022-11-04T22:54:09Z,CONTRIBUTOR,,"My understand is that `INSERT OR IGNORE` will ignore when inserts would cause duplicate keys so I'm not sure exactly why the error is raised from `sqlite3`.
```
import argparse
from pathlib import Path
from xklb import db, utils
from xklb.utils import log
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument(""database"")
parser.add_argument(""dbs"", nargs=""*"")
parser.add_argument(""--upsert"")
parser.add_argument(""--db"", ""-db"", help=argparse.SUPPRESS)
parser.add_argument(""--verbose"", ""-v"", action=""count"", default=0)
args = parser.parse_args()
if args.db:
args.database = args.db
Path(args.database).touch()
args.db = db.connect(args)
log.info(utils.dict_filter_bool(args.__dict__))
return args
def merge_db(args, source_db):
source_db = str(Path(source_db).resolve())
s_db = db.connect(argparse.Namespace(database=source_db, verbose=args.verbose))
for table in [s for s in s_db.table_names() if not ""_fts"" in s and not s.startswith(""sqlite_"")]:
log.info(""[%s]: %s"", source_db, table)
with s_db.conn:
data = s_db[table].rows
with args.db.conn:
if args.upsert:
args.db[table].upsert_all(data, pk=args.upsert.split("",""), alter=True)
else:
args.db[table].insert_all(data, alter=True, replace=True)
def merge_dbs():
args = parse_args()
for s_db in args.dbs:
merge_db(args, s_db)
if __name__ == ""__main__"":
merge_dbs()
```
```
$ lb-dev merge video.db tube_71.db --upsert path -vv
SQL: INSERT OR IGNORE INTO [media]([path]) VALUES(?); - params: ['https://archive.org/details/088ghostofachanceroygetssackedrevengeofthelivinglunchdvdripxvidphz']
...
File ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:3122, in Table.insert_all(self, records, pk, foreign_keys, column_order, not_null, defaults, batch_size, hash_id, hash_id_columns, alter, ignore, replace, truncate, extracts, conversions, columns, upsert, analyze)
3116 all_columns += [
3117 column for column in record if column not in all_columns
3118 ]
3120 first = False
-> 3122 self.insert_chunk(
3123 alter,
3124 extracts,
3125 chunk,
3126 all_columns,
3127 hash_id,
3128 hash_id_columns,
3129 upsert,
3130 pk,
3131 conversions,
3132 num_records_processed,
3133 replace,
3134 ignore,
3135 )
3137 if analyze:
3138 self.analyze()
File ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:2887, in Table.insert_chunk(self, alter, extracts, chunk, all_columns, hash_id, hash_id_columns, upsert, pk, conversions, num_records_processed, replace, ignore)
2885 for query, params in queries_and_params:
2886 try:
-> 2887 result = self.db.execute(query, params)
2888 except OperationalError as e:
2889 if alter and ("" column"" in e.args[0]):
2890 # Attempt to add any missing columns, then try again
File ~/.local/lib/python3.10/site-packages/sqlite_utils/db.py:484, in Database.execute(self, sql, parameters)
482 self._tracer(sql, parameters)
483 if parameters is not None:
--> 484 return self.conn.execute(sql, parameters)
485 else:
486 return self.conn.execute(sql)
IntegrityError: constraint failed
> /home/xk/.local/lib/python3.10/site-packages/sqlite_utils/db.py(484)execute()
482 self._tracer(sql, parameters)
483 if parameters is not None:
--> 484 return self.conn.execute(sql, parameters)
485 else:
486 return self.conn.execute(sql)
```
```
sqlite3 --version
3.36.0 2021-06-18 18:36:39
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/511/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1431786951,I_kwDOBm6k_c5VV1XH,1876,SQL query should wrap on SQL interrupted screen,9599,closed,0,,,2,2022-11-01T17:14:01Z,2022-11-01T17:22:33Z,2022-11-01T17:22:33Z,OWNER,,"Just saw this:
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1876/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1423182778,I_kwDOCGYnMM5U1Au6,505,Release sqlite-utils 3.30,9599,closed,0,,,2,2022-10-25T22:20:05Z,2022-10-25T22:41:26Z,2022-10-25T22:41:16Z,OWNER,,https://github.com/simonw/sqlite-utils/compare/3.29...defa2974c6d3abc19be28d6b319649b8028dc966,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/505/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1420055377,I_kwDOBm6k_c5UpFNR,1847,Both _local_metadata and _metadata_local?,9599,closed,0,,,2,2022-10-24T01:43:08Z,2022-10-24T01:53:13Z,2022-10-24T01:53:13Z,OWNER,,"Spotted this in the debugger against the `datasette` object while running tests (`pytest -k test_permissions_cascade` to be exact):
```
(Pdb) [p for p in dir(self) if p.startswith('_') and '__' not in p]
['_actor', '_asset_urls', '_connected_databases', '_crumb_items', '_local_metadata', '_metadata', '_metadata_local', '_metadata_recursive_update', '_permission_checks', '_plugins', '_prepare_connection', '_refresh_schemas', '_refresh_schemas_lock', '_register_custom_units', '_register_renderers', '_root_token', '_routes', '_secret', '_settings', '_show_messages', '_startup_hook_calculation', '_startup_hook_fired', '_startup_invoked', '_threads', '_versions', '_write_messages_to_response']
```
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1847/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1397193691,I_kwDOBm6k_c5TR3vb,1832,__bool__ method on Results,9599,closed,0,,,2,2022-10-05T04:18:12Z,2022-10-05T04:32:33Z,2022-10-05T04:32:33Z,OWNER,,"Wrote this code today: https://github.com/simonw/datasette-public/blob/1401bfae50e71c1dfd2bfb6954f2e86d5a7ab21b/datasette_public/__init__.py#L41
```python
results = await db.execute(
""select 1 from _public_tables where table_name = ?"", [table_name]
)
if len(results):
return True
```
Would be nice if I could use `if results` there instead.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1832/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1393903845,I_kwDOBm6k_c5TFUjl,1828,word-wrap: anywhere resulting in weird display,9599,closed,0,,,2,2022-10-02T21:25:03Z,2022-10-02T23:01:17Z,2022-10-02T23:01:17Z,OWNER,,"e.g. on https://github-to-sqlite.dogsheep.net/github/commits
This is from a change introduced here: https://github.com/simonw/datasette/commit/bf8d84af5422606597be893cedd375020cb2b369 in #1805
https://github.com/simonw/datasette/blob/bf8d84af5422606597be893cedd375020cb2b369/datasette/static/app.css#L447-L450",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1828/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1386593843,I_kwDOCGYnMM5Spb4z,494,Document how to use Just,9599,closed,0,,,2,2022-09-26T19:25:12Z,2022-09-26T19:32:36Z,2022-09-26T19:26:39Z,OWNER,,"I'm using `just` a lot know, based on this file - I should add that to https://sqlite-utils.datasette.io/en/latest/contributing.html
https://github.com/simonw/sqlite-utils/blob/afbd2b2cba45cccb305c3d4638d18db4dd3d4bbd/Justfile#L1-L24",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/494/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1363765916,I_kwDOCGYnMM5RSWqc,483,`sqlite-utils install` command,9599,closed,0,,,2,2022-09-06T20:13:55Z,2022-09-26T19:04:43Z,2022-09-26T18:57:15Z,OWNER,,"With the addition of `--functions` in:
- #471
In addition to the existing `convert` command, there are now very good reasons to want to install additional packages into the same virtual environment as `sqlite-utils` itself, to allow them to be used with those features.
This isn't easy if you installed the tool with `pipx` or `brew install sqlite-utils`.
Datasette solved this problem with the `datasette install` command:
- https://github.com/simonw/datasette/issues/925
`sqlite-utils` could benefit from the same idea.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/483/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
520508502,MDU6SXNzdWU1MjA1MDg1MDI=,31,"""friends"" command (similar to ""followers"")",9599,closed,0,,,2,2019-11-09T20:20:20Z,2022-09-20T05:05:03Z,2020-02-07T07:03:28Z,MEMBER,,"Current list of commands:
```
followers Save followers for specified user (defaults to...
followers-ids Populate followers table with IDs of account followers
friends-ids Populate followers table with IDs of account friends
```
Obvious omission here is `friends`, which would be powered by `https://api.twitter.com/1.1/friends/list.json`: https://developer.twitter.com/en/docs/accounts-and-users/follow-search-get-users/api-reference/get-friends-list",206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/31/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1378640768,I_kwDOBm6k_c5SLGOA,1816,Validate settings.json on startup in configuration directory mode,9599,closed,0,,,2,2022-09-19T23:35:18Z,2022-09-20T01:15:48Z,2022-09-20T01:15:48Z,OWNER,,"> It might have been useful for Datasette to show an error when started against a `settings.json` file that contains an invalid setting though.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1814#issuecomment-1251677554_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1816/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1378495690,I_kwDOBm6k_c5SKizK,1814,Static files not served,4068,closed,0,,,2,2022-09-19T20:38:17Z,2022-09-19T23:35:06Z,2022-09-19T23:34:30Z,NONE,,"Folder structure:
```
bibliography/
bibliography/static-files
bibliography/static-files/styles.css
bibliography/bibliography.db
bibliography/metadata.json
bibliography/settings.json
```
```
$ cat bibliography/settings.json
{
""suggest_facets"": false,
""truncate_cells_html"": 1000,
""static"": ""assets:static-files/""
}
```
File `/assets/styles.css` is not found (HTTP 404, `Database not found: assets`).
Using datasette revision d0737e4de51ce178e556fc011ccb8cc46bbb6359.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1814/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1367835380,I_kwDOCGYnMM5Rh4L0,487,Specify foreign key against compound key in other table,540968,closed,0,,,2,2022-09-09T13:32:09Z,2022-09-11T04:00:44Z,2022-09-11T04:00:44Z,NONE,,"When inserting rows via the library, is it possible to specify a foreign key to a compound primary key?
For example, suppose I create a table:
```
db = Database('events.db')
db['events'].insert_all([
{'venue': 'Times Square', 'date': '2022-12-31', 'title': 'Rockin New Year Eve'},
{'venue': 'Wembley Stadium', 'date': '2022-06-05', 'title': 'FA Cup'},
{'venue': 'Times Square', 'date': '2021-12-31', 'title': 'Rockin New Year Eve'},
], pk=('date', 'venue'))
```
And I want to add related data in another table:
```
act = {'name': 'Rick Astley', 'venue': 'Times Square', 'date': '2021-12-31' }
db['performers'].insert(act, pk=??>)
```
Is it possible to specify a value for `pk` that will point to the compound primary key in `events`?
SQLite does support it:
https://www.sqlite.org/foreignkeys.html#fk_composite",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/487/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1353441389,I_kwDOCGYnMM5Qq-Bt,477,Conda Forge,49702524,closed,0,,,2,2022-08-28T19:03:08Z,2022-09-07T03:46:55Z,2022-09-07T03:46:55Z,NONE,,"Hello! I have successfully put this package on to Conda Forge, and I have extending the invitation for the owner/maintainers of this package to be maintainers on Conda Forge as well. Let me know if you are interested! Thanks.
https://github.com/conda-forge/sqlite-utils-feedstock",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/477/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1353196970,I_kwDOCGYnMM5QqCWq,476,Release notes for 3.29,9599,closed,0,,8355157,2,2022-08-27T23:21:21Z,2022-08-28T04:07:15Z,2022-08-28T04:07:03Z,OWNER,,https://github.com/simonw/sqlite-utils/compare/3.28...104f37fa4d2e7e5999c1d829267b62c737f74d3e,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/476/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1352946135,I_kwDOCGYnMM5QpFHX,472,Reuse the locals/globals fix from --functions for other code accepting options,9599,closed,0,,8355157,2,2022-08-27T05:12:05Z,2022-08-27T05:20:12Z,2022-08-27T05:20:12Z,OWNER,,"I figured out a workaround for the ugly `global x` hack here:
- https://github.com/simonw/sqlite-utils/issues/471#issuecomment-1229120653",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/472/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1339663518,I_kwDOBm6k_c5P2aSe,1784,"Include ""entrypoint"" option on `--load-extension`?",15178711,closed,0,,,2,2022-08-16T00:22:57Z,2022-08-23T18:34:31Z,2022-08-23T18:34:31Z,CONTRIBUTOR,,"## Problem
SQLite extensions have the option to define multiple ""entrypoints"" in each loadable extension. For example, the upcoming version of `sqlite-lines` will have 2 entrypoints: the default `sqlite3_lines_init` (which SQLite will automatically guess for) and `sqlite3_lines_noread_init`. The `sqlite3_lines_noread_init` version omits functions that read from the filesystem, which is necessary for security purposes when running untrusted SQL (which Datasette does).
(Similar multiple entrypoints will also be added for sqlite-http).
The `--load-extension` flag, however, doesn't give the option to specify a different entrypoint, so the default one is always used.
## Proposal
I want there to be a new command line option of the `--load-extension` flag to specify a custom entrypoint like so:
```
datasette my.db \
--load-extension ./lines0 sqlite3_lines0_noread_init
```
Then, under the hood, this line of code:
https://github.com/simonw/datasette/blob/7af67b54b7d9bca43e948510fc62f6db2b748fa8/datasette/app.py#L562
Would look something like this:
```python
conn.execute(""SELECT load_extension(?, ?)"", [extension, entrypoint])
```
One potential problem: For backward compatibility, I'm not sure if Click allows cli flags to have variable number of options (""arity""). So I guess it could also use a `:` delimiter like `--static`:
```
datasette my.db \
--load-extension ./lines0:sqlite3_lines0_noread_init
```
Or maybe even a new flag name?
```
datasette my.db \
--load-extension-entrypoint ./lines0 sqlite3_lines0_noread_init
```
Personally I prefer the `:` option... and maybe even `--load-extension` -> `--load`? Definitely out of scope for this issue tho
```
datasette my.db \
--load./lines0:sqlite3_lines0_noread_init
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1784/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1338278056,I_kwDOBm6k_c5PxICo,1782,Release notes for Datasette 0.62,9599,closed,0,,8303187,2,2022-08-14T15:26:45Z,2022-08-14T17:40:45Z,2022-08-14T17:32:54Z,OWNER,,"I've written a lot of these already for the alphas:
- https://github.com/simonw/datasette/releases/tag/0.62a0
- https://github.com/simonw/datasette/releases/tag/0.62a1",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1782/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1223527226,I_kwDOBm6k_c5I7Ys6,1738,"""Cannot use _sort and _sort_desc at the same time""",9599,closed,0,,8303187,2,2022-05-03T01:06:24Z,2022-08-14T16:13:55Z,2022-08-14T16:13:55Z,OWNER,,"Triggered this error while playing with the sort desc checkbox and the apply button that are only visible on this page at mobile screen width:
https://latest.datasette.io/fixtures/compound_three_primary_keys?_sort_desc=pk1
Navigate to that page (with the browser narrow enough to show the box), un-check the box and click Apply:
![sort-bug](https://user-images.githubusercontent.com/9599/166390804-cb289b29-63dc-4986-b7f9-81cf2ae04914.gif)
Also notable: I managed to get to a page with `?_sort_desk=pk1` in the URL three times by clicking around with that button.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1738/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1298531653,I_kwDOCGYnMM5NZgVF,451,Make sqlite_utils.utils.chunks a documented function,9599,closed,0,,,2,2022-07-08T06:01:04Z,2022-07-15T22:09:34Z,2022-07-15T21:59:33Z,OWNER,,I want to use it in another project: https://github.com/simonw/sqlite-utils/blob/8a9fe6498faf783a1fdeb1793e661ad194a05267/sqlite_utils/utils.py#L471-L474,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/451/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1277328147,I_kwDOCGYnMM5MInsT,446,Use Just to automate running tests and linters locally,9599,closed,0,,,2,2022-06-20T19:51:09Z,2022-06-21T19:28:35Z,2022-06-20T19:54:50Z,OWNER,,I keep committing code that fails additional tests like `mypy` and `flake8` and `black`. Automate those using Just.,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/446/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1269998342,I_kwDOCGYnMM5LsqMG,443,Make `utils.rows_from_file()` a documented API,9599,closed,0,,,2,2022-06-13T21:53:24Z,2022-06-20T19:49:37Z,2022-06-14T20:12:46Z,OWNER,,"> `rows_from_file()` isn't part of the documented API but maybe it should be!
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/440#issuecomment-1154385916_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/443/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1269886084,I_kwDOCGYnMM5LsOyE,442,`maximize_csv_field_size_limit()` utility function,9599,closed,0,,,2,2022-06-13T19:54:54Z,2022-06-14T21:55:15Z,2022-06-14T21:31:49Z,OWNER,,"This code here runs only if `cli.py` is imported: https://github.com/simonw/sqlite-utils/blob/7ddf5300886a32d6daf60cf1d71efe492b65c87e/sqlite_utils/cli.py#L50-L59
I found myself needing the same fix in another library:
- https://github.com/simonw/datasette-socrata/issues/13
It should be a documented utility function.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/442/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1250161887,I_kwDOCGYnMM5Kg_Tf,438,illegal UTF-16 surrogate,4068,closed,0,,,2,2022-05-26T22:49:52Z,2022-05-27T08:21:53Z,2022-05-27T08:21:53Z,NONE,,"I am trying to insert `https://artsdatabanken.no/Fab2018/api/export/csv` into a SQLite database, but I have an error when using `sqlite-utils`:
```
sqlite-utils insert --csv --delimiter "";"" --encoding=""utf-16-le"" --pk ""Id"" csv fremmedart test.db
[------------------------------------] 0%
Error: 'utf-16-le' codec can't decode bytes in position 98-99: illegal UTF-16 surrogate
The input you provided uses a character encoding other than utf-8.
You can fix this by passing the --encoding= option with the encoding of the file.
If you do not know the encoding, running 'file filename.csv' may tell you.
It's often worth trying: --encoding=latin-1
```
I tried to convert the file using `iconv -f ""utf-16le"" -t ""utf-8""`, but I still get a similar error (slightly different position):
```
sqlite-utils insert --csv --delimiter "";"" --encoding=utf-8 --pk ""Id"" csv_utf8 fremmedart test.db
[------------------------------------] 0%
Error: 'utf-8' codec can't decode byte 0xd9 in position 99: invalid continuation byte
The input you provided uses a character encoding other than utf-8.
You can fix this by passing the --encoding= option with the encoding of the file.
If you do not know the encoding, running 'file filename.csv' may tell you.
It's often worth trying: --encoding=latin-1
```
I have no issues reading such file using this Python code:
```python
content = open('csv', encoding='utf-16-le').read())
```
`in2csv` works too.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/438/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1243704847,I_kwDOCGYnMM5KIW4P,435,Switch to Furo documentation theme,9599,closed,0,,,2,2022-05-20T21:46:39Z,2022-05-20T21:56:10Z,2022-05-20T21:54:43Z,OWNER,,"As seen in:
- https://github.com/simonw/datasette/issues/1746
- https://github.com/simonw/shot-scraper/issues/77",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/435/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1243517592,I_kwDOBm6k_c5KHpKY,1748,Add copy buttons next to code examples in the documentation,9599,closed,0,,,2,2022-05-20T19:09:00Z,2022-05-20T19:15:00Z,2022-05-20T19:11:32Z,OWNER,,Similar to the ones in `datasette-copyable` which are implemented here: https://github.com/executablebooks/sphinx-copybutton/tree/f84c001a0507f8ec46779d0701b079a265564583,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1748/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1171599874,I_kwDOCGYnMM5F1TIC,415,Convert with `--multi` and `--dry-run` flag does not work,3976183,closed,0,,,2,2022-03-16T21:59:46Z,2022-03-21T04:18:24Z,2022-03-21T04:18:24Z,NONE,,"It's not possible to combine `--multi` and `--dry-run` flag in the `convert` command.
Let's first create a simple database from JSON string
```console
$ echo '[{""foo"": ""abc""}]' | sqlite-utils insert demo.db demo -
$ sqlite-utils query demo.db ""SELECT * FROM demo""
[{""foo"": ""abc""}]
```
and then try to convert the ""foo"" column with a static value ""bar"" (see docs [Converting a column into multiple columns](https://sqlite-utils.datasette.io/en/stable/cli.html#converting-a-column-into-multiple-columns))
```console
$ sqlite-utils convert demo.db demo foo '{""foo"": ""bar""}' --multi --dry-run
Traceback (most recent call last):
File ""/home/dotcs/anaconda3/envs/tools/bin/sqlite-utils"", line 8, in
sys.exit(cli())
File ""/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py"", line 1128, in __call__
return self.main(*args, **kwargs)
File ""/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py"", line 1053, in main
rv = self.invoke(ctx)
File ""/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py"", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py"", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/click/core.py"", line 754, in invoke
return __callback(*args, **kwargs)
File ""/home/dotcs/anaconda3/envs/tools/lib/python3.9/site-packages/sqlite_utils/cli.py"", line 2686, in convert
for row in db.conn.execute(sql, where_args).fetchall():
sqlite3.OperationalError: user-defined function raised exception
```
But without the `--dry-run` flag it does work as expected:
```console
$ sqlite-utils convert demo.db demo foo '{""foo"": ""bar""}' --multi
$ sqlite-utils query demo.db ""SELECT * FROM demo""
[{""foo"": ""bar""}]
```
```console
$ sqlite-utils --version
sqlite-utils, version 3.25.1
```",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/415/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1174404647,I_kwDOBm6k_c5F__4n,1669,Release 0.61 alpha,9599,closed,0,,,2,2022-03-20T00:35:35Z,2022-03-20T01:24:36Z,2022-03-20T01:24:36Z,OWNER,,"> I'm going to release this as a 0.61 alpha so I can more easily depend on it from `datasette-hashed-urls`.
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1668#issuecomment-1073136896_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1669/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1170554975,I_kwDOBm6k_c5FxUBf,1663,Document the internals that were used in datasette-hashed-urls,9599,closed,0,,3268330,2,2022-03-16T05:17:08Z,2022-03-19T04:04:50Z,2022-03-17T21:32:38Z,OWNER,,"The https://github.com/simonw/datasette-hashed-urls used a couple of currently undocumented features:
- `db.hash`
- `Datasette(..., immutables=[...])`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1663/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1122416919,I_kwDOBm6k_c5C5rkX,1623,/-/patterns returns link: alternate JSON header to 404,9599,closed,0,,3268330,2,2022-02-02T21:42:49Z,2022-03-19T04:04:49Z,2022-02-02T21:48:56Z,OWNER,,"Bug from:
- #1620
```
% curl -s -I 'https://latest.datasette.io/-/patterns' | grep link
link: https://latest.datasette.io/-/patterns.json; rel=""alternate""; type=""application/json+datasette""
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1623/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1088816961,I_kwDODEm0Qs5A5gdB,62,KeyError: 'created_at' for private accounts?,6764957,closed,0,,,2,2021-12-26T17:51:51Z,2022-03-12T02:36:32Z,2022-02-24T18:10:18Z,NONE,,"hey Simon!
i was running `twitter-to-sqlite user-timeline twitter.db` for [my private alt](https://twitter.com/swyxio) and ran into this error:
![image](https://user-images.githubusercontent.com/6764957/147416165-46b69c30-100a-406f-8534-8612b75547ae.png)
```bash
Traceback (most recent call last):
File ""/Users/swyx/Work/datasette/env/bin/twitter-to-sqlite"", line 8, in
sys.exit(cli())
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py"", line 1128, in __call__
return self.main(*args, **kwargs)
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py"", line 1053, in main
rv = self.invoke(ctx)
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py"", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py"", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/click/core.py"", line 754, in invoke
return __callback(*args, **kwargs)
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/cli.py"", line 291, in user_timeline
profile = utils.get_profile(db, session, **kwargs)
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py"", line 133, in get_profile
save_users(db, [profile])
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py"", line 453, in save_users
transform_user(user)
File ""/Users/swyx/Work/datasette/env/lib/python3.9/site-packages/twitter_to_sqlite/utils.py"", line 285, in transform_user
user[""created_at""] = parser.parse(user[""created_at""])
KeyError: 'created_at'
```
this looks awfully like #37 but it can't be, because i'm authed into my account and obviously i have perms to read my own account. wonder if there's any diagnostic methods i should apply here? just filing an issue for others to find while i investigate.",206156866,issue,,,"{""url"": ""https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/62/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1128139375,I_kwDOCGYnMM5DPgpv,405,"`Database(memory_name=""name"")` constructor argument",9599,closed,0,,,2,2022-02-09T07:15:03Z,2022-02-16T01:23:16Z,2022-02-16T01:23:16Z,OWNER,,"SQLite in-memory databases can be named, in which case multiple connections can be opened to a shared in-memory database running within the same process.
Datasette supports this - SQLite could support it too.
https://docs.datasette.io/en/0.60.2/internals.html#database-ds-path-none-is-mutable-false-is-memory-false-memory-name-none",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/405/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1128120451,I_kwDOCGYnMM5DPcCD,404,Add example of `--convert` to the help for `sqlite-utils insert`,9599,closed,0,,,2,2022-02-09T06:49:09Z,2022-02-09T06:56:35Z,2022-02-09T06:55:16Z,OWNER,,"https://sqlite-utils.datasette.io/en/3.23/cli-reference.html#insert would be more useful if it included an example of `--convert` in action.
I can maybe use an example from https://simonwillison.net/2022/Jan/11/sqlite-utils/",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/404/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1109783030,I_kwDOBm6k_c5CJfH2,1607,More detailed information about installed SpatiaLite version,9599,closed,0,,3268330,2,2022-01-20T21:28:03Z,2022-02-09T06:42:02Z,2022-02-09T06:32:28Z,OWNER,,"https://www.gaia-gis.it/gaia-sins/spatialite-sql-5.0.0.html#version has a whole bunch of interesting functions for things like `freexl_version()` and `geos_version()` and `HasMathSQL()` and suchlike.
These could be shown on the `/-/versions` page.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1607/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1125081640,I_kwDOCGYnMM5DD2Io,401,Update SpatiaLite example in the documentation,9599,closed,0,,,2,2022-02-06T02:02:07Z,2022-02-06T02:05:03Z,2022-02-06T02:03:24Z,OWNER,,"This one here: https://sqlite-utils.datasette.io/en/3.23/python-api.html#converting-column-values-using-sql-functions
It should take advantage of the new methods from:
- #79",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/401/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
723460107,MDU6SXNzdWU3MjM0NjAxMDc=,187,Maybe: Utility method / CLI tool for initializing SpatiaLite,9599,closed,0,,,2,2020-10-16T19:04:03Z,2022-02-05T00:04:26Z,2020-10-16T19:15:13Z,OWNER,,"> I think this should initialize SpatiaLite against the current database if it has not been initialized already.
>
> Relevant code: https://github.com/simonw/shapefile-to-sqlite/blob/e754d0747ca2facf9a7433e2d5d15a6a37a9cf6e/shapefile_to_sqlite/utils.py#L112-L126",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/187/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
557825032,MDU6SXNzdWU1NTc4MjUwMzI=,77,Ability to insert data that is transformed by a SQL function,9599,closed,0,,,2,2020-01-30T23:45:55Z,2022-02-05T00:04:25Z,2020-01-31T00:24:32Z,OWNER,,"I want to be able to run the equivalent of this SQL insert:
```python
# Convert to ""Well Known Text"" format
wkt = shape(geojson['geometry']).wkt
# Insert and commit the record
conn.execute(""INSERT INTO places (id, name, geom) VALUES(null, ?, GeomFromText(?, 4326))"", (
""Wales"", wkt
))
conn.commit()
```
From the Datasette SpatiaLite docs: https://datasette.readthedocs.io/en/stable/spatialite.html
To do this, I need a way of telling `sqlite-utils` that a specific column should be wrapped in `GeomFromText(?, 4326)`.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/77/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1117132741,I_kwDOBm6k_c5ClhfF,1615,Potential simplified publishing mechanism,369053,closed,0,,,2,2022-01-28T08:34:50Z,2022-02-02T07:34:21Z,2022-02-02T07:34:17Z,NONE,,"Hi,
Forewarning: this idea is one I've only been thinking about for a while and it's not fully fleshed-out yet.
I love Datasette and what it stands for. I was thinking about how we could make it accessible to more people, especially those without access to credit cards required for a lot of hosting options. Or they might not feel comfortable signing up for said services.
So I was thinking I might create a service that hosts Datasette instances for folks. I'd probably stick it on AWS Lambda and limit requests to something like n/month to avoid bankrupting myself. If I did build such a hypothetical service, I was thinking I would rely on GitHub Actions to do the heavy lifting.
E.g. user `johndoe` creates a repo `my-animals` with a couple of files: `dogs.csv`, `cats.csv` and the following GitHub Actions workflow:
```yaml
# .github/workflows/push.yml
on:
push
# this allows the publish action to use OIDC to authenticate johndoe/my-animals
permissions:
id-token: write
contents: read
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python@v2
- run: pip install sqlite-utils
- uses: actions/checkout@v2
- run: |
set -eux
sqlite-utils create-database animals.db
sqlite-utils insert animals.db dogs dogs.csv --csv
sqlite-utils insert animals.db cats cats.csv --csv
- uses: datasette-hub/publish@v1
with:
db: animals.db
metadata: meta.yml
# this step is helpful for debugging why the
# generated sqlite db was rejected
- uses: actions/upload-artifact@v2
if: failure()
with:
path: animals.db
retention-days: 1
```
This would then cause a Datasette instance to be available at `https://johndoe-my-animals.datasette-hub.test/`. It feels like this could significantly reduce the friction to someone being able to go from data set to Datasette.
What do you think? Does this address a real need? Or am I perhaps misunderstanding the main friction points? As a bonus: it feels like this would pair well with [git scraping](https://simonwillison.net/2020/Oct/9/git-scraping/).",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1615/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1114638930,I_kwDOCGYnMM5CcApS,391,`sqlite-utils bulk` progress bar,9599,closed,0,,,2,2022-01-26T05:14:49Z,2022-01-26T05:17:20Z,2022-01-26T05:16:51Z,OWNER,,"It can easily have a progress bar because it works by looping through an iterator: https://github.com/simonw/sqlite-utils/blob/a9fca7efa4184fbb2a65ca1275c326950ed9d3c1/sqlite_utils/cli.py#L1014-L1018
Should also support the `--silent` option if I add this.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/391/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
471818939,MDU6SXNzdWU0NzE4MTg5Mzk=,48,"Jupyter notebook demo of the library, launchable on Binder",9599,closed,0,,,2,2019-07-23T17:05:05Z,2022-01-26T02:08:46Z,2022-01-26T02:08:39Z,OWNER,,,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/48/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1114544727,I_kwDOCGYnMM5CbppX,389,Plausible analytics for documentation,9599,closed,0,,,2,2022-01-26T01:58:35Z,2022-01-26T02:07:41Z,2022-01-26T02:07:41Z,OWNER,,"```html
```
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/388#issuecomment-1021785268_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/389/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
842416110,MDU6SXNzdWU4NDI0MTYxMTA=,1278,SpatiaLite timezones demo is broken,9599,closed,0,,,2,2021-03-27T04:45:27Z,2022-01-20T21:29:43Z,2021-03-27T16:17:13Z,OWNER,,https://github.com/simonw/datasette/blob/5fd02890650db790b2ffdb90eb9f78f8e0639c37/docs/spatialite.rst#L96,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1278/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
443040665,MDU6SXNzdWU0NDMwNDA2NjU=,466,"Move ""no such module: VirtualSpatialIndex"" code elsewhere",9599,closed,0,,4305096,2,2019-05-11T22:09:00Z,2022-01-20T21:29:41Z,2019-05-11T22:57:22Z,OWNER,,"We currently show a useful warning (from #331) when the user tries to open a spatialite database without first loading the module:
https://github.com/simonw/datasette/blob/c692cd291111050483a32bea1ee08e994a0b781b/datasette/app.py#L547-L554
This code is part of `.inspect()` which is going away - see #462 - so I need to find somewhere else for it to live.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/466/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
563347679,MDU6SXNzdWU1NjMzNDc2Nzk=,668,Make it easier to load SpatiaLite,9599,closed,0,,,2,2020-02-11T17:03:43Z,2022-01-20T21:29:41Z,2021-01-04T20:18:39Z,OWNER,,"```
$ datasette spatial.db
Serve! files=('spatial.db',) (immutables=()) on port 8001
ERROR: conn=, sql = 'PRAGMA table_info(SpatialIndex);', params = None: no such module: VirtualSpatialIndex
Usage: datasette serve [OPTIONS] [FILES]...
Error: It looks like you're trying to load a SpatiaLite database without first loading the SpatiaLite module.
Read more: https://datasette.readthedocs.io/en/latest/spatialite.html
```
This error message could sniff around in the common locations for the SpatiaLite module and output the CLI command you should use to enable it:
```
datasette spatial.db --load-extension=/usr/local/lib/mod_spatialite.dylib
```
Even better: if Datasette had a `--spatialite` option which automatically loads the extension from common locations, if it can find it.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/668/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
336936010,MDU6SXNzdWUzMzY5MzYwMTA=,331,Datasette throws error when loading spatialite db without extension loaded,82988,closed,0,,,2,2018-06-29T09:51:14Z,2022-01-20T21:29:40Z,2018-07-10T15:13:36Z,CONTRIBUTOR,,"When starting datasette on a SpatialLite database *without* loading the SpatiaLite extension (using eg `--load-extension=/usr/local/lib/mod_spatialite.dylib`) an error is thrown and the server fails to start:
```
datasette -p 8003 adminboundaries.db
Serve! files=('adminboundaries.db',) on port 8003
Traceback (most recent call last):
File ""/Users/ajh59/anaconda3/bin/datasette"", line 11, in
sys.exit(cli())
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 722, in __call__
return self.main(*args, **kwargs)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 697, in main
rv = self.invoke(ctx)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/click/core.py"", line 535, in invoke
return callback(*args, **kwargs)
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/cli.py"", line 552, in serve
ds.inspect()
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/app.py"", line 273, in inspect
""tables"": inspect_tables(conn, self.metadata.get(""databases"", {}).get(name, {}))
File ""/Users/ajh59/anaconda3/lib/python3.6/site-packages/datasette/inspect.py"", line 79, in inspect_tables
""PRAGMA table_info({});"".format(escape_sqlite(table))
sqlite3.OperationalError: no such module: VirtualSpatialIndex
```
It would be nice to trap this and return a message saying something like:
```
It looks like you're trying to load a SpatiaLite database? Make sure you load in the SpatiaLite extension when starting datasette.
Read more: https://datasette.readthedocs.io/en/latest/spatialite.html
```
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/331/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1097040427,I_kwDOBm6k_c5BY4Ir,1587,Add `sqlite_stat1`(-4) tables to hidden table list,9599,closed,0,,,2,2022-01-08T21:28:20Z,2022-01-20T04:12:59Z,2022-01-20T04:12:59Z,OWNER,,"> Running `ANALYZE` creates a new visible table called `sqlite_stat1`: https://www.sqlite.org/fileformat.html#the_sqlite_stat1_table
>
> This should be added to the default list of hidden tables in Datasette.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1587/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
991467558,MDU6SXNzdWU5OTE0Njc1NTg=,1466,Add Datasette Desktop to installation documentation,9599,closed,0,,7571612,2,2021-09-08T19:41:27Z,2022-01-13T22:28:28Z,2022-01-13T21:55:18Z,OWNER,,See https://datasette.io/desktop and https://simonwillison.net/2021/Sep/8/datasette-desktop/,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1466/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083921371,I_kwDOBm6k_c5Am1Pb,1570,Separate db.execute_write() into three methods,9599,closed,0,,7571612,2,2021-12-18T18:45:54Z,2022-01-13T22:27:38Z,2021-12-18T18:57:25Z,OWNER,,"> Rather than adding a `executemany=True` parameter, I'm now thinking a better design might be to have three methods:
>
> - `db.execute_write(sql, params=None, block=False)`
> - `db.execute_write_script(sql, block=False)`
> - `db.execute_write_many(sql, params_seq, block=False)`
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997267416_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1570/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083895395,I_kwDOBm6k_c5Amu5j,1569,"db.execute_write(..., executescript=True) parameter",9599,closed,0,,7571612,2,2021-12-18T18:20:47Z,2022-01-13T22:27:27Z,2021-12-18T18:34:18Z,OWNER,,"> Idea: teach `execute_write` to accept an optional `executescript=True` parameter, like this:
```diff
diff --git a/datasette/database.py b/datasette/database.py
index 468e936..1a424f5 100644
--- a/datasette/database.py
+++ b/datasette/database.py
@@ -94,10 +94,14 @@ class Database:
f""file:{self.path}{qs}"", uri=True, check_same_thread=False
)
- async def execute_write(self, sql, params=None, block=False):
+ async def execute_write(self, sql, params=None, executescript=False, block=False):
+ assert not executescript and params, ""Cannot use params with executescript=True""
def _inner(conn):
with conn:
- return conn.execute(sql, params or [])
+ if executescript:
+ return conn.executescript(sql)
+ else:
+ return conn.execute(sql, params or [])
with trace(""sql"", database=self.name, sql=sql.strip(), params=params):
results = await self.execute_write_fn(_inner, block=block)
```
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997248364_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1569/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083726550,I_kwDOBm6k_c5AmFrW,1568,Trace should show queries on the write connection too,9599,closed,0,,7571612,2,2021-12-18T02:34:12Z,2022-01-13T22:27:23Z,2021-12-18T02:42:34Z,OWNER,,"> Here's why - `trace` only applies to read, not write SQL operations: https://github.com/simonw/datasette/blob/7c8f8aa209e4ba7bf83976f8495d67c28fbfca24/datasette/database.py#L209-L211
_Originally posted by @simonw in https://github.com/simonw/datasette/issues/1555#issuecomment-997128508_",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1568/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1083573206,I_kwDOBm6k_c5AlgPW,1563,Datasette(... files=) should not be a required argument,9599,closed,0,,7571612,2,2021-12-17T19:54:18Z,2022-01-13T22:27:18Z,2021-12-18T02:19:40Z,OWNER,,"```pycon
>>> ds = Datasette(memory=True)
Traceback (most recent call last):
File """", line 1, in
TypeError: __init__() missing 1 required positional argument: 'files'
>>> ds = Datasette(memory=True, files=[])
```
I wanted to create an in-memory Datasette for running some tests, no point in forcing me to pass `files=[]` to do that.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1563/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1099584685,I_kwDOCGYnMM5BilSt,381,`sqlite-utils rows` options `--limit` and `--offset`,9599,closed,0,,,2,2022-01-11T20:23:12Z,2022-01-11T23:33:37Z,2022-01-11T23:19:36Z,OWNER,,Because I often want to use it just to preview a few rows from the database. Piping through `| head -n 20` works for JSON and CSV (they stream) but not for `--table`.,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/381/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1073712378,I_kwDOBm6k_c4__4z6,1544,Code that detects the label column for a table is case-sensitive,9599,closed,0,,,2,2021-12-07T20:01:25Z,2021-12-07T20:03:43Z,2021-12-07T20:03:43Z,OWNER,,I just noticed that a column called `Name` is not being picked up as the label column for a table.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1544/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1066501534,I_kwDOCGYnMM4_kYWe,345,`table.strict` introspection boolean for identifying STRICT mode tables,9599,closed,0,,,2,2021-11-29T21:05:10Z,2021-11-29T22:45:26Z,2021-11-29T22:44:36Z,OWNER,,"> From the STRICT docs:
>> The SQLite parser accepts a comma-separated list of table options after the final close parenthesis in a CREATE TABLE statement. As of this writing (2021-08-23) only two options are recognized:
>>
>> - STRICT
>> - [WITHOUT ROWID](https://www.sqlite.org/withoutrowid.html)
>
> So I think I need to read the `CREATE TABLE` statement from the `sqlite_master` table, split on the last `)`, split those tokens on `,` and see if `create` is in there (case insensitive).
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982020757_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/345/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1059549523,I_kwDOBm6k_c4_J3FT,1526,"Add to vercel.json, rather than overwriting it.",192568,closed,0,,,2,2021-11-22T00:47:12Z,2021-11-22T04:49:45Z,2021-11-22T04:13:47Z,CONTRIBUTOR,,"I'd like to be able to add to vercel.json. But Datasette overwrites whatever I put in that file. I originally reported this here:
https://github.com/simonw/datasette-publish-vercel/issues/51
In that case, I wanted to do a rewrite... and now I need to do 301 redirects (because we had to rename our site).
Can this be addressed?
",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1526/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1057996111,I_kwDOBm6k_c4_D71P,1517,Let `register_routes()` over-ride default routes within Datasette,9599,closed,0,,3268330,2,2021-11-19T00:22:15Z,2021-11-19T03:20:00Z,2021-11-19T03:07:27Z,OWNER,,"See https://github.com/simonw/datasette/issues/878#issuecomment-973554024_ - right now `register_routes()` can't replace default Datasette routes.
It would be neat if plugins could do this - especially if there was a neat documented way for them to then re-dispatch to the original route code after making some kind of modification.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1517/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1026794056,I_kwDOCGYnMM49M6JI,331,Mypy error: found module but no type hints or library stubs,53032010,closed,0,,,2,2021-10-14T20:29:50Z,2021-11-14T23:21:08Z,2021-11-14T23:21:08Z,NONE,,"```
Python 3.9.5
mypy 0.910
sqlite-utils 3.17.1
```
While using sqlite-utils as a library, when I use mypy for static type checking, it throws an error:
```
mypy .
src/etl.py:5: error: Skipping analyzing ""sqlite_utils"": found module but no type hints or library stubs
import sqlite_utils
^
src/etl.py:5: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
test/test_etl.py:4: error: Skipping analyzing ""sqlite_utils"": found module but no type hints or library stubs
import sqlite_utils
^
Found 2 errors in 2 files (checked 7 source files)
```
When I add a `py.typed` file to the sqlite-utils package to mark it as PEP 561 compatible, the error goes away.
```
al@nbal ..b/python3.9/site-packages/sqlite_utils (git)-[main] % la
total 200
drwx------ 3 al al 4096 Oct 14 22:00 .
drwx------ 117 al al 4096 Oct 12 21:12 ..
-rw------- 1 al al 64409 Oct 12 21:11 cli.py
-rw------- 1 al al 109092 Oct 12 21:11 db.py
-rw------- 1 al al 0 Oct 14 22:00 py.typed
-rw------- 1 al al 684 Oct 12 21:11 recipes.py
-rw------- 1 al al 7988 Oct 12 21:11 utils.py
-rw------- 1 al al 113 Oct 12 21:11 __init__.py
```
I would like to suggest adding a `py.typed` file to the repository.
See also the mypy docs on creating PEP 561 compatible packages:
https://mypy.readthedocs.io/en/stable/installed_packages.html#creating-pep-561-compatible-packages
",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/331/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
1021550542,I_kwDOBm6k_c4845_O,1482,Support Python 3.10,9599,closed,0,,,2,2021-10-09T00:30:52Z,2021-10-24T22:21:40Z,2021-10-24T22:19:55Z,OWNER,,"I started work on this in #1481 where I found a Python 3.10 bug that needs a workaround in Janus, see:
- https://github.com/aio-libs/janus/issues/358
This is a tracking issue for anything else that shows up.
This is also needed for the Homebrew package to upgrade to 3.10:
- https://github.com/Homebrew/homebrew-core/pull/86932",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1482/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
994450961,MDU6SXNzdWU5OTQ0NTA5NjE=,1469,"Column cog shows ""facet by this"" when already default faceted",9599,closed,0,,,2,2021-09-13T04:51:26Z,2021-10-13T21:20:07Z,2021-10-13T21:20:07Z,OWNER,,"e.g. on https://covid-19.datasettes.com/covid/economist_excess_deaths
But if you add `?_facet=country` to the URL that goes away: https://covid-19.datasettes.com/covid/economist_excess_deaths?_facet_size=5&_facet=country
The logic that decides if the ""Facet by this"" item is shown does not take default `metadata.json` facets into account.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1469/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
994390593,MDU6SXNzdWU5OTQzOTA1OTM=,1468,Faceting for custom SQL queries,72577720,closed,0,,,2,2021-09-13T02:52:16Z,2021-09-13T04:54:22Z,2021-09-13T04:54:17Z,CONTRIBUTOR,,"Facets are awesome. But not when I need to join to tidy tables together. Or even just running explicitly the default SQL query that simply lists all the rows and columns of a table (up to SIZE). That is to say, when I browse a table, I see facets:
https://latest.datasette.io/fixtures/compound_three_primary_keys
But when I run a custom query, I don't:
https://latest.datasette.io/fixtures?sql=select+pk1%2C+pk2%2C+pk3%2C+content+from+compound_three_primary_keys+order+by+pk1%2C+pk2%2C+pk3+limit+101
Is there an idiom to cause custom SQL to come back with facet suggestions?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1468/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
976405225,MDU6SXNzdWU5NzY0MDUyMjU=,320,sqlite-utils memory --analyze option,9599,closed,0,,,2,2021-08-22T15:37:10Z,2021-08-22T15:46:56Z,2021-08-22T15:44:29Z,OWNER,,To provide a way of running [analyze-tables](https://sqlite-utils.datasette.io/en/stable/cli.html#analyzing-tables) directly against JSON or CSV data.,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/320/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
965210966,MDU6SXNzdWU5NjUyMTA5NjY=,314,Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__`,9599,closed,0,,,2,2021-08-10T18:03:59Z,2021-08-18T22:25:21Z,2021-08-18T22:25:21Z,OWNER,,"> Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue.
_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/312#issuecomment-896200682_",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/314/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
951581763,MDU6SXNzdWU5NTE1ODE3NjM=,298,Read lines with JSON object,2172260,closed,0,,,2,2021-07-23T13:28:52Z,2021-08-03T06:50:47Z,2021-08-02T21:55:16Z,NONE,,"I found this posted on HN a while ago and love it -- thank you!
As a minor improvement, it would be great to have the ability to parse a file with line-separated JSON objects. Currently the parser obviously requires an array wrapping all these objects.",140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/298/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
958516743,MDU6SXNzdWU5NTg1MTY3NDM=,306,Configure sphinx.ext.extlinks for issues,9599,closed,0,,,2,2021-08-02T21:19:19Z,2021-08-02T21:39:34Z,2021-08-02T21:29:22Z,OWNER,,As seen in Datasette: https://github.com/simonw/datasette/issues/1227,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/306/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
957298475,MDU6SXNzdWU5NTcyOTg0NzU=,1407,OSError: AF_UNIX path too long in ds_unix_domain_socket_server,9599,closed,0,,,2,2021-07-31T18:36:06Z,2021-07-31T19:03:44Z,2021-07-31T19:03:44Z,OWNER,,"Got this exception while working on #1406.
```
@pytest.fixture(scope=""session"")
def ds_unix_domain_socket_server(tmp_path_factory):
socket_folder = tmp_path_factory.mktemp(""uds"")
uds = str(socket_folder / ""datasette.sock"")
ds_proc = subprocess.Popen(
[""datasette"", ""--memory"", ""--uds"", uds],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
cwd=tempfile.gettempdir(),
)
# Give the server time to start
time.sleep(1.5)
# Check it started successfully
> assert not ds_proc.poll(), ds_proc.stdout.read().decode(""utf-8"")
E AssertionError: INFO: Started server process [48453]
E INFO: Waiting for application startup.
E INFO: Application startup complete.
E Traceback (most recent call last):
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/bin/datasette"", line 33, in
E sys.exit(load_entry_point('datasette', 'console_scripts', 'datasette')())
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1137, in __call__
E return self.main(*args, **kwargs)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1062, in main
E rv = self.invoke(ctx)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1668, in invoke
E return _process_result(sub_ctx.command.invoke(sub_ctx))
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 1404, in invoke
E return ctx.invoke(self.callback, **ctx.params)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/click/core.py"", line 763, in invoke
E return __callback(*args, **kwargs)
E File ""/Users/simon/Dropbox/Development/datasette/datasette/cli.py"", line 583, in serve
E uvicorn.run(ds.app(), **uvicorn_kwargs)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/main.py"", line 393, in run
E server.run()
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 50, in run
E loop.run_until_complete(self.serve(sockets=sockets))
E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py"", line 616, in run_until_complete
E return future.result()
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 67, in serve
E await self.startup(sockets=sockets)
E File ""/Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.8/site-packages/uvicorn/server.py"", line 133, in startup
E server = await asyncio.start_unix_server(
E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/streams.py"", line 132, in start_unix_server
E return await loop.create_unix_server(factory, path, **kwds)
E File ""/Users/simon/.pyenv/versions/3.8.2/lib/python3.8/asyncio/unix_events.py"", line 296, in create_unix_server
E sock.bind(path)
E OSError: AF_UNIX path too long
E
E assert not 1
E + where 1 = >()
E + where > = .poll
```",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1407/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
940891698,MDU6SXNzdWU5NDA4OTE2OTg=,1390,Mention restarting systemd in documentation,9599,closed,0,,,2,2021-07-09T16:05:15Z,2021-07-09T16:32:57Z,2021-07-09T16:32:33Z,OWNER,,"https://docs.datasette.io/en/stable/deploying.html#running-datasette-using-systemd
Need to clarify that if you add a new database or change metadata you need to restart systemd.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1390/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
927766296,MDU6SXNzdWU5Mjc3NjYyOTY=,291,Adopt flake8,9599,closed,0,,,2,2021-06-23T01:19:37Z,2021-06-24T17:50:27Z,2021-06-24T17:50:27Z,OWNER,,,140912432,issue,,,"{""url"": ""https://api.github.com/repos/simonw/sqlite-utils/issues/291/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
503190241,MDU6SXNzdWU1MDMxOTAyNDE=,584,Codec error in some CSV exports,9599,closed,0,,,2,2019-10-07T01:15:34Z,2021-06-17T18:13:20Z,2019-10-18T05:23:16Z,OWNER,,"Got this exploring my Swarm checkins:
![448DBFC4-71F8-4846-83C0-BEA511B2157A](https://user-images.githubusercontent.com/9599/66279259-3af53480-e865-11e9-9651-04fd2d895392.jpeg)
`/swarm/stickers.csv?stickerType=messageOnly&_size=max`",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/584/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
516748849,MDU6SXNzdWU1MTY3NDg4NDk=,612,CSV export is broken for tables with null foreign keys,9599,closed,0,,,2,2019-11-02T22:52:47Z,2021-06-17T18:13:20Z,2019-11-02T23:12:53Z,OWNER,,"Following on from #406 - this CSV export appears to be broken:
https://14da705.datasette.io/fixtures/foreign_key_references.csv?_labels=on&_size=max
```csv
pk,foreign_key_with_label,foreign_key_with_label_label,foreign_key_with_no_label,foreign_key_with_no_label_label
1,1,hello,1,1
2,,
```
That second row should have 5 values, but it only has 4.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/612/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
759695780,MDU6SXNzdWU3NTk2OTU3ODA=,1133,Option to omit header row in CSV export,9599,closed,0,,,2,2020-12-08T18:54:46Z,2021-06-17T18:12:31Z,2020-12-10T23:28:51Z,OWNER,,`?_header=off` - for symmetry with existing option `?_nl=on`.,107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1133/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
919508498,MDU6SXNzdWU5MTk1MDg0OTg=,1375,JSON export dumps JSON fields as TEXT,4068,closed,0,,,2,2021-06-12T09:45:08Z,2021-06-14T09:41:59Z,2021-06-13T15:37:58Z,NONE,,"Hi!
When a user tries to export data as JSON, I would expect to see the value of JSON columns represented as JSON instead of being rendered as a string. What do you think?",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1375/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed
912464443,MDU6SXNzdWU5MTI0NjQ0NDM=,1360,"Security flaw, to be fixed in 0.56.1 and 0.57",9599,closed,0,,,2,2021-06-05T21:53:51Z,2021-06-05T22:23:23Z,2021-06-05T22:22:06Z,OWNER,,"See security advisory here for details: https://github.com/simonw/datasette/security/advisories/GHSA-xw7c-jx9m-xh5g - the `?_trace=1` debugging option was not correctly escaping its JSON output, resulting in a [reflected cross-site scripting](https://owasp.org/www-community/attacks/xss/#reflected-xss-attacks) vulnerability.",107914493,issue,,,"{""url"": ""https://api.github.com/repos/simonw/datasette/issues/1360/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed