(impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1303?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [0a7621f...c348ff1](https://codecov.io/gh/simonw/datasette/pull/1303?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",861331159,
https://github.com/simonw/datasette/issues/1300#issuecomment-821971059,https://api.github.com/repos/simonw/datasette/issues/1300,821971059,MDEyOklzc3VlQ29tbWVudDgyMTk3MTA1OQ==,3243482,2021-04-18T10:42:19Z,2021-04-18T10:42:19Z,CONTRIBUTOR,"If there's a simpler way to generate a URL for a specific row, I'm all ears","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",860625833,
https://github.com/simonw/datasette/issues/1300#issuecomment-821970965,https://api.github.com/repos/simonw/datasette/issues/1300,821970965,MDEyOklzc3VlQ29tbWVudDgyMTk3MDk2NQ==,3243482,2021-04-18T10:41:15Z,2021-04-18T10:41:15Z,CONTRIBUTOR,"If I change the hookspec and add a row parameter, it works
https://github.com/simonw/datasette/blob/7a2ed9f8a119e220b66d67c7b9e07cbab47b1196/datasette/hookspecs.py#L58
```
def render_cell(value, column, row, table, database, datasette):
```
But to generate a URL, I need the primary keys, but I can't call `pks = await db.primary_keys(table)` inside a sync function. I can't call `datasette.utils.detect_primary_keys` either, because the db connection is not publicly exposed (AFAICT).
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",860625833,
https://github.com/simonw/datasette/issues/1196#issuecomment-819775388,https://api.github.com/repos/simonw/datasette/issues/1196,819775388,MDEyOklzc3VlQ29tbWVudDgxOTc3NTM4OA==,1219001,2021-04-14T19:28:38Z,2021-04-14T19:28:38Z,NONE,@QAInsights I'm having a similar problem when publishing to Cloud Run on Windows. It's not able to access certain packages in my conda environment where Datasette is installed. Can you explain how you got it to work in WSL? Were you able to access the .db file in the Windows file system? Thank you.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",791237799,
https://github.com/simonw/datasette/pull/1296#issuecomment-819467759,https://api.github.com/repos/simonw/datasette/issues/1296,819467759,MDEyOklzc3VlQ29tbWVudDgxOTQ2Nzc1OQ==,295329,2021-04-14T12:07:37Z,2021-04-14T12:11:36Z,CONTRIBUTOR,"> Removing /var/lib/apt and /var/lib/dpkg makes apt and dpkg unusable in
images based on this one. Running `apt-get clean` and removing
/var/lib/apt/lists achieves similar size savings.
this PR helps me as removing the /var/lib/apt and /var/lib/dpkg directories breaks my ability to add packages when using `datasetteproject/datasette:0.56` as a base image.
----
Shorterm workaround for me was to use this in my Dockerfile
```
FROM datasetteproject/datasette:0.56
RUN mkdir -p /var/lib/apt
RUN mkdir -p /var/lib/dpkg
RUN mkdir -p /var/lib/dpkg/updates
RUN mkdir -p /var/lib/dpkg/info
RUN touch /var/lib/dpkg/status
RUN apt-get update # and install your packages etc
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",855446829,
https://github.com/simonw/datasette/issues/830#issuecomment-817414881,https://api.github.com/repos/simonw/datasette/issues/830,817414881,MDEyOklzc3VlQ29tbWVudDgxNzQxNDg4MQ==,192568,2021-04-12T01:06:34Z,2021-04-12T01:07:27Z,CONTRIBUTOR,"Related: #1285, including arguments for natural breaks, equal interval, etc. modeled after choropleth map legends.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",636511683,
https://github.com/simonw/datasette/issues/1295#issuecomment-817301355,https://api.github.com/repos/simonw/datasette/issues/1295,817301355,MDEyOklzc3VlQ29tbWVudDgxNzMwMTM1NQ==,9599,2021-04-11T12:40:25Z,2021-04-11T12:41:06Z,OWNER,"I could have a page about error codes in the docs, then have `https://datasette.io/E123` style URLs for each error core which are shown when that error occurs and redirect to the corresponding documentation section.
Can enforce these with a documentation unit test.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",855296937,
https://github.com/simonw/datasette/issues/1286#issuecomment-815978405,https://api.github.com/repos/simonw/datasette/issues/1286,815978405,MDEyOklzc3VlQ29tbWVudDgxNTk3ODQwNQ==,192568,2021-04-08T16:47:29Z,2021-04-10T03:59:00Z,CONTRIBUTOR,"This worked for me:
`| {{ cell.value | replace('"", ""','; ') | replace('[\""','') | replace('\""]','')}} | `
I'm sure there is a prettier (and more flexible) way, but for now, this is ever-so-much more pleasant to look at.
------ AFTER:
![]()
------ BEFORE:
![]()
(Note: I didn't figure out how to have one item have no semicolon, while multi-items close with a semicolon, but this is good enough for now. I also didn't figure out how to set up a new jinja filter. I don't want to add to /datasette/utils/__init__.py as I assume that would get overwritten when upgrading datasette. Having a starter guide on creating jinja filters in datasette would be helpful. (The jinja documentation isn't datasette-specific enough for me to quite nail it.)
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849220154,
https://github.com/simonw/datasette/issues/1293#issuecomment-813480043,https://api.github.com/repos/simonw/datasette/issues/1293,813480043,MDEyOklzc3VlQ29tbWVudDgxMzQ4MDA0Mw==,9599,2021-04-05T16:16:17Z,2021-04-05T16:16:17Z,OWNER,"https://latest.datasette.io/fixtures?sql=explain+select+*+from+paginated_view will be an interesting test query - because `paginated_view` is defined like this:
```sql
CREATE VIEW paginated_view AS
SELECT
content,
'- ' || content || ' -' AS content_extra
FROM no_primary_key;
```
So this will help test that the mechanism isn't confused by output columns that are created through a concatenation expression.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813445512,https://api.github.com/repos/simonw/datasette/issues/1293,813445512,MDEyOklzc3VlQ29tbWVudDgxMzQ0NTUxMg==,9599,2021-04-05T15:11:40Z,2021-04-05T15:11:40Z,OWNER,"Here's some older example code that works with opcodes from Python, in this case to output indexes used by a query: https://github.com/plasticityai/supersqlite/blob/master/supersqlite/idxchk.py","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813438771,https://api.github.com/repos/simonw/datasette/issues/1293,813438771,MDEyOklzc3VlQ29tbWVudDgxMzQzODc3MQ==,9599,2021-04-05T14:58:48Z,2021-04-05T14:58:48Z,OWNER,I may need to do something special for rowid columns - there is a `RowId` opcode that might come into play here.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/dogsheep/dogsheep-photos/issues/35#issuecomment-813249000,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/35,813249000,MDEyOklzc3VlQ29tbWVudDgxMzI0OTAwMA==,1151557,2021-04-05T07:37:57Z,2021-04-05T07:37:57Z,NONE,"There are trained ML models used in Photoprism:
- https://dl.photoprism.org/tensorflow/nasnet.zip
- https://dl.photoprism.org/tensorflow/nsfw.zip","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",842695374,
https://github.com/simonw/datasette/issues/620#issuecomment-813167335,https://api.github.com/repos/simonw/datasette/issues/620,813167335,MDEyOklzc3VlQ29tbWVudDgxMzE2NzMzNQ==,9599,2021-04-05T03:57:22Z,2021-04-05T03:57:22Z,OWNER,This may be obsoleted by #1293 - it looks like I may be able to auto-detect these foreign keys for arbitrary queries after all.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",520667773,
https://github.com/simonw/datasette/issues/1293#issuecomment-813164282,https://api.github.com/repos/simonw/datasette/issues/1293,813164282,MDEyOklzc3VlQ29tbWVudDgxMzE2NDI4Mg==,9599,2021-04-05T03:42:26Z,2021-04-05T03:42:36Z,OWNER,"Extracting variables with this trick appears to work OK, but you have to pass the correct variables to the `explain select...` query. Using `defaultdict` seems to work there:
```pycon
>>> rows = conn.execute('explain select * from repos where id = :id', defaultdict(int))
>>> [dict(r) for r in rows if r['opcode'] == 'Variable']
[{'addr': 2,
'opcode': 'Variable',
'p1': 1,
'p2': 1,
'p3': 0,
'p4': ':id',
'p5': 0,
'comment': None}]
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813162622,https://api.github.com/repos/simonw/datasette/issues/1293,813162622,MDEyOklzc3VlQ29tbWVudDgxMzE2MjYyMg==,9599,2021-04-05T03:34:24Z,2021-04-05T03:40:35Z,OWNER,"This almost works, but throws errors with some queries (anything with a `rowid` column for example) - it needs a bunch of test coverage.
```python
def columns_for_query(conn, sql):
rows = conn.execute('explain ' + sql).fetchall()
table_rootpage_by_register = {r['p1']: r['p2'] for r in rows if r['opcode'] == 'OpenRead'}
names_by_rootpage = dict(
conn.execute(
'select rootpage, name from sqlite_master where rootpage in ({})'.format(
', '.join(map(str, table_rootpage_by_register.values()))
)
)
)
columns_by_column_register = {}
for row in rows:
if row['opcode'] == 'Column':
addr, opcode, table_id, cid, column_register, p4, p5, comment = row
table = names_by_rootpage[table_rootpage_by_register[table_id]]
columns_by_column_register[column_register] = (table, cid)
result_row = [dict(r) for r in rows if r['opcode'] == 'ResultRow'][0]
registers = list(range(result_row[""p1""], result_row[""p1""] + result_row[""p2""] - 1))
all_column_names = {}
for table in names_by_rootpage.values():
table_xinfo = conn.execute('pragma table_xinfo({})'.format(table)).fetchall()
for row in table_xinfo:
all_column_names[(table, row[""cid""])] = row[""name""]
final_output = []
for r in registers:
try:
table, cid = columns_by_column_register[r]
final_output.append((table, all_column_names[table, cid]))
except KeyError:
final_output.append((None, None))
return final_output
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813134637,https://api.github.com/repos/simonw/datasette/issues/1293,813134637,MDEyOklzc3VlQ29tbWVudDgxMzEzNDYzNw==,9599,2021-04-05T01:21:59Z,2021-04-05T01:21:59Z,OWNER,"http://www.sqlite.org/draft/lang_explain.html says:
> Applications should not use EXPLAIN or EXPLAIN QUERY PLAN since their exact behavior is variable and only partially documented.
I'm going to keep exploring this though.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813134227,https://api.github.com/repos/simonw/datasette/issues/1293,813134227,MDEyOklzc3VlQ29tbWVudDgxMzEzNDIyNw==,9599,2021-04-05T01:19:31Z,2021-04-05T01:19:31Z,OWNER,"| addr | opcode | p1 | p2 | p3 | p4 | p5 | comment |
|--------|---------------|------|------|------|-----------------------|------|-----------|
| 0 | Init | 0 | 47 | 0 | | 00 | |
| 1 | OpenRead | 0 | 51 | 0 | 15 | 00 | |
| 2 | Integer | 15 | 2 | 0 | | 00 | |
| 3 | Once | 0 | 15 | 0 | | 00 | |
| 4 | OpenEphemeral | 2 | 1 | 0 | k(1,) | 00 | |
| 5 | VOpen | 1 | 0 | 0 | vtab:3E692C362158 | 00 | |
| 6 | String8 | 0 | 5 | 0 | CPAD_2020a_SuperUnits | 00 | |
| 7 | SCopy | 7 | 6 | 0 | | 00 | |
| 8 | Integer | 2 | 3 | 0 | | 00 | |
| 9 | Integer | 2 | 4 | 0 | | 00 | |
| 10 | VFilter | 1 | 15 | 3 | | 00 | |
| 11 | Rowid | 1 | 8 | 0 | | 00 | |
| 12 | MakeRecord | 8 | 1 | 9 | C | 00 | |
| 13 | IdxInsert | 2 | 9 | 8 | 1 | 00 | |
| 14 | VNext | 1 | 11 | 0 | | 00 | |
| 15 | Return | 2 | 0 | 0 | | 00 | |
| 16 | Rewind | 2 | 46 | 0 | | 00 | |
| 17 | Column | 2 | 0 | 1 | | 00 | |
| 18 | IsNull | 1 | 45 | 0 | | 00 | |
| 19 | SeekRowid | 0 | 45 | 1 | | 00 | |
| 20 | Column | 0 | 2 | 11 | | 00 | |
| 21 | Function0 | 1 | 10 | 9 | like(2) | 02 | |
| 22 | IfNot | 9 | 45 | 1 | | 00 | |
| 23 | Column | 0 | 14 | 13 | | 00 | |
| 24 | Function0 | 1 | 12 | 9 | intersects(2) | 02 | |
| 25 | Ne | 14 | 45 | 9 | | 51 | |
| 26 | Column | 0 | 14 | 9 | | 00 | |
| 27 | Function0 | 0 | 9 | 15 | asgeojson(1) | 01 | |
| 28 | Rowid | 0 | 16 | 0 | | 00 | |
| 29 | Column | 0 | 1 | 17 | | 00 | |
| 30 | Column | 0 | 2 | 18 | | 00 | |
| 31 | Column | 0 | 3 | 19 | | 00 | |
| 32 | Column | 0 | 4 | 20 | | 00 | |
| 33 | Column | 0 | 5 | 21 | | 00 | |
| 34 | Column | 0 | 6 | 22 | | 00 | |
| 35 | Column | 0 | 7 | 23 | | 00 | |
| 36 | Column | 0 | 8 | 24 | | 00 | |
| 37 | Column | 0 | 9 | 25 | | 00 | |
| 38 | Column | 0 | 10 | 26 | | 00 | |
| 39 | Column | 0 | 11 | 27 | | 00 | |
| 40 | RealAffinity | 27 | 0 | 0 | | 00 | |
| 41 | Column | 0 | 12 | 28 | | 00 | |
| 42 | Column | 0 | 13 | 29 | | 00 | |
| 43 | Column | 0 | 14 | 30 | | 00 | |
| 44 | ResultRow | 15 | 16 | 0 | | 00 | |
| 45 | Next | 2 | 17 | 0 | | 00 | |
| 46 | Halt | 0 | 0 | 0 | | 00 | |
| 47 | Transaction | 0 | 0 | 265 | 0 | 01 | |
| 48 | Variable | 1 | 31 | 0 | :freedraw | 00 | |
| 49 | Function0 | 1 | 31 | 7 | geomfromgeojson(1) | 01 | |
| 50 | String8 | 0 | 10 | 0 | %mini% | 00 | |
| 51 | Variable | 1 | 32 | 0 | :freedraw | 00 | |
| 52 | Function0 | 1 | 32 | 12 | geomfromgeojson(1) | 01 | |
| 53 | Integer | 1 | 14 | 0 | | 00 | |
| 54 | Goto | 0 | 1 | 0 | | 00 | |
Essential documentation for understanding that output: https://www.sqlite.org/opcode.html","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813134072,https://api.github.com/repos/simonw/datasette/issues/1293,813134072,MDEyOklzc3VlQ29tbWVudDgxMzEzNDA3Mg==,9599,2021-04-05T01:18:37Z,2021-04-05T01:18:37Z,OWNER,"Had a fantastic suggestion on the SQLite forum: it might be possible to get what I want by interpreting the opcodes output by `explain select ...`.
Copying the reply I posted to this thread:
That's really useful, thanks! It looks like it _might_ be possible for me to reconstruct where each column came from using the `explain select` output.
Here's a complex example:
It looks like the opcodes I need to inspect are `OpenRead`, `Column` and `ResultRow`.
`OpenRead` tells me which tables are being opened - the `p2` value (in this case 51) corresponds to the `rootpage` column in `sqlite_master` here: - it gets assigned to the register in `p1`.
The `Column` opcodes tell me which columns are being read - `p1` is that table reference, and `p2` is the `cid` of the column within that table.
The `ResultRow` opcode then tells me which columns are used in the results. `15 16` means start at the 15th and then read the next 16 columns.
I think this might work!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813116177,https://api.github.com/repos/simonw/datasette/issues/1293,813116177,MDEyOklzc3VlQ29tbWVudDgxMzExNjE3Nw==,9599,2021-04-04T23:31:00Z,2021-04-04T23:31:00Z,OWNER,"Sadly it doesn't do what I need. This query should only return one column, but instead I get back every column that was consulted by the query:
![]()
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813115607,https://api.github.com/repos/simonw/datasette/issues/1293,813115607,MDEyOklzc3VlQ29tbWVudDgxMzExNTYwNw==,9599,2021-04-04T23:25:15Z,2021-04-04T23:25:15Z,OWNER,"Oh wow, I just spotted https://github.com/macbre/sql-metadata
> Uses tokenized query returned by python-sqlparse and generates query metadata. Extracts column names and tables used by the query. Provides a helper for normalization of SQL queries and tables aliases resolving.
It's for MySQL, PostgreSQL and Hive right now but maybe getting it working with SQLite wouldn't be too hard?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813115414,https://api.github.com/repos/simonw/datasette/issues/1293,813115414,MDEyOklzc3VlQ29tbWVudDgxMzExNTQxNA==,9599,2021-04-04T23:23:34Z,2021-04-04T23:23:34Z,OWNER,"The other approach I considered for this was to have my own SQL query parser running in Python, which could pick apart a complex query and figure out which column was sourced from which table. I dropped this idea because it felt that the moment `select *` came into play a pure parsing approach wouldn't work - I'd need knowledge of the schema in order to resolve the `*`.
A Python parser approach might be good enough to handle a subset of queries - those that don't use `select *` for example - and maybe that would be worth shipping? The feature doesn't have to be perfect for it to be useful.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813114933,https://api.github.com/repos/simonw/datasette/issues/1293,813114933,MDEyOklzc3VlQ29tbWVudDgxMzExNDkzMw==,9599,2021-04-04T23:19:22Z,2021-04-04T23:19:22Z,OWNER,I asked about this on the SQLite forum: https://sqlite.org/forum/forumpost/0180277fb7,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813113653,https://api.github.com/repos/simonw/datasette/issues/1293,813113653,MDEyOklzc3VlQ29tbWVudDgxMzExMzY1Mw==,9599,2021-04-04T23:10:49Z,2021-04-04T23:10:49Z,OWNER,"One option I've not fully explored yet: could I write my own custom SQLite C extension which exposes this functionality as a callable function?
Then I could load that extension and run a SQL query something like this:
```
select database, table, column from analyze_query(:sql_query)
```
Where `analyze_query(...)` would be a fancy virtual table function of some sort that uses the underlying `sqlite3_column_database_name()` C functions to analyze the SQL query and return details of what it would return.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813113403,https://api.github.com/repos/simonw/datasette/issues/1293,813113403,MDEyOklzc3VlQ29tbWVudDgxMzExMzQwMw==,9599,2021-04-04T23:08:48Z,2021-04-04T23:08:48Z,OWNER,"Worth noting that adding `limit 0` to the query still causes it to conduct the permission checks, hopefully while avoiding doing any of the actual work of executing the query:
```pycon
In [20]: db.execute('select * from compound_primary_key join facetable on facetable.rowid = compound_primary_key.rowid limit 0').fetchall()
...:
args (21, None, None, None, None) kwargs {}
args (20, 'compound_primary_key', 'pk1', 'main', None) kwargs {}
args (20, 'compound_primary_key', 'pk2', 'main', None) kwargs {}
args (20, 'compound_primary_key', 'content', 'main', None) kwargs {}
args (20, 'facetable', 'pk', 'main', None) kwargs {}
args (20, 'facetable', 'created', 'main', None) kwargs {}
args (20, 'facetable', 'planet_int', 'main', None) kwargs {}
args (20, 'facetable', 'on_earth', 'main', None) kwargs {}
args (20, 'facetable', 'state', 'main', None) kwargs {}
args (20, 'facetable', 'city_id', 'main', None) kwargs {}
args (20, 'facetable', 'neighborhood', 'main', None) kwargs {}
args (20, 'facetable', 'tags', 'main', None) kwargs {}
args (20, 'facetable', 'complex_array', 'main', None) kwargs {}
args (20, 'facetable', 'distinct_some_null', 'main', None) kwargs {}
args (20, 'facetable', 'pk', 'main', None) kwargs {}
args (20, 'compound_primary_key', 'ROWID', 'main', None) kwargs {}
Out[20]: []
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813113218,https://api.github.com/repos/simonw/datasette/issues/1293,813113218,MDEyOklzc3VlQ29tbWVudDgxMzExMzIxOA==,9599,2021-04-04T23:07:25Z,2021-04-04T23:07:25Z,OWNER,"Here are all of the available constants:
```pycon
In [3]: for k in dir(sqlite3):
...: if k.startswith(""SQLITE_""):
...: print(k, getattr(sqlite3, k))
...:
SQLITE_ALTER_TABLE 26
SQLITE_ANALYZE 28
SQLITE_ATTACH 24
SQLITE_CREATE_INDEX 1
SQLITE_CREATE_TABLE 2
SQLITE_CREATE_TEMP_INDEX 3
SQLITE_CREATE_TEMP_TABLE 4
SQLITE_CREATE_TEMP_TRIGGER 5
SQLITE_CREATE_TEMP_VIEW 6
SQLITE_CREATE_TRIGGER 7
SQLITE_CREATE_VIEW 8
SQLITE_CREATE_VTABLE 29
SQLITE_DELETE 9
SQLITE_DENY 1
SQLITE_DETACH 25
SQLITE_DONE 101
SQLITE_DROP_INDEX 10
SQLITE_DROP_TABLE 11
SQLITE_DROP_TEMP_INDEX 12
SQLITE_DROP_TEMP_TABLE 13
SQLITE_DROP_TEMP_TRIGGER 14
SQLITE_DROP_TEMP_VIEW 15
SQLITE_DROP_TRIGGER 16
SQLITE_DROP_VIEW 17
SQLITE_DROP_VTABLE 30
SQLITE_FUNCTION 31
SQLITE_IGNORE 2
SQLITE_INSERT 18
SQLITE_OK 0
SQLITE_PRAGMA 19
SQLITE_READ 20
SQLITE_RECURSIVE 33
SQLITE_REINDEX 27
SQLITE_SAVEPOINT 32
SQLITE_SELECT 21
SQLITE_TRANSACTION 22
SQLITE_UPDATE 23
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813113175,https://api.github.com/repos/simonw/datasette/issues/1293,813113175,MDEyOklzc3VlQ29tbWVudDgxMzExMzE3NQ==,9599,2021-04-04T23:07:01Z,2021-04-04T23:07:01Z,OWNER,"A more promising route I found involved the `db.set_authorizer` method. This can be used to log the permission checks that SQLite uses, including checks for permission to access specific columns of specific tables. For a while I thought this could work!
```pycon
>>> def print_args(*args, **kwargs):
... print(""args"", args, ""kwargs"", kwargs)
... return sqlite3.SQLITE_OK
>>> db = sqlite3.connect(""fixtures.db"")
>>> db.execute('select * from compound_primary_key join facetable on rowid').fetchall()
args (21, None, None, None, None) kwargs {}
args (20, 'compound_primary_key', 'pk1', 'main', None) kwargs {}
args (20, 'compound_primary_key', 'pk2', 'main', None) kwargs {}
args (20, 'compound_primary_key', 'content', 'main', None) kwargs {}
args (20, 'facetable', 'pk', 'main', None) kwargs {}
args (20, 'facetable', 'created', 'main', None) kwargs {}
args (20, 'facetable', 'planet_int', 'main', None) kwargs {}
args (20, 'facetable', 'on_earth', 'main', None) kwargs {}
args (20, 'facetable', 'state', 'main', None) kwargs {}
args (20, 'facetable', 'city_id', 'main', None) kwargs {}
args (20, 'facetable', 'neighborhood', 'main', None) kwargs {}
args (20, 'facetable', 'tags', 'main', None) kwargs {}
args (20, 'facetable', 'complex_array', 'main', None) kwargs {}
args (20, 'facetable', 'distinct_some_null', 'main', None) kwargs {}
```
Those `20` values (where 20 is `SQLITE_READ`) looked like they were checking permissions for the columns in the order they would be returned!
Then I found a snag:
```pycon
In [18]: db.execute('select 1 + 1 + (select max(rowid) from facetable)')
args (21, None, None, None, None) kwargs {}
args (31, None, 'max', None, None) kwargs {}
args (20, 'facetable', 'pk', 'main', None) kwargs {}
args (21, None, None, None, None) kwargs {}
args (20, 'facetable', '', None, None) kwargs {}
```
Once a subselect is involved the order of the `20` checks no longer matches the order in which the columns are returned from the query.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,
https://github.com/simonw/datasette/issues/1293#issuecomment-813112546,https://api.github.com/repos/simonw/datasette/issues/1293,813112546,MDEyOklzc3VlQ29tbWVudDgxMzExMjU0Ng==,9599,2021-04-04T23:02:45Z,2021-04-04T23:02:45Z,OWNER,"I've done various pieces of research into this over the past few years. Capturing what I've discovered in this ticket.
The SQLite C API has functions that can help with this: https://www.sqlite.org/c3ref/column_database_name.html details those. But they're not exposed in the Python SQLite library.
Maybe it would be possible to use them via `ctypes`? My hunch is that I would have to re-implement the full `sqlite3` module with `ctypes`, which sounds daunting.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",849978964,