html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app https://github.com/simonw/datasette/issues/1871#issuecomment-1299349741,https://api.github.com/repos/simonw/datasette/issues/1871,1299349741,IC_kwDOBm6k_c5NcoDt,9599,simonw,2022-11-01T23:22:55Z,2022-11-01T23:22:55Z,OWNER,"It's weird that the API explorer only lets you explore POST APIs. It should probably also let you explore GET APIs, or be renamed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1427293909,API explorer tool, https://github.com/simonw/datasette/issues/1879#issuecomment-1299098458,https://api.github.com/repos/simonw/datasette/issues/1879,1299098458,IC_kwDOBm6k_c5Nbqta,9599,simonw,2022-11-01T20:27:40Z,2022-11-01T20:33:52Z,OWNER,"https://github.com/simonw/datasette-x-forwarded-host/blob/main/datasette_x_forwarded_host/__init__.py could happen in core controlled by: `--setting trust_forwarded_host 1`","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432037325,Make it easier to fix URL proxy problems, https://github.com/simonw/datasette/issues/1879#issuecomment-1299102108,https://api.github.com/repos/simonw/datasette/issues/1879,1299102108,IC_kwDOBm6k_c5Nbrmc,9599,simonw,2022-11-01T20:30:54Z,2022-11-01T20:33:06Z,OWNER,One idea: add a `/-/debug` page (or `/-/tips` or `/-/checks`) which shows the incoming requests headers and could even detect if there's an `x-forwarded-host` header that isn't being repeated and show a tip on how to fix that.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432037325,Make it easier to fix URL proxy problems, https://github.com/simonw/datasette/issues/1879#issuecomment-1299102755,https://api.github.com/repos/simonw/datasette/issues/1879,1299102755,IC_kwDOBm6k_c5Nbrwj,9599,simonw,2022-11-01T20:31:37Z,2022-11-01T20:31:37Z,OWNER,And some JavaScript that can spot if Datasette thinks it is being served over HTTP when it's actually being served over HTTPS.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432037325,Make it easier to fix URL proxy problems, https://github.com/simonw/datasette/issues/1879#issuecomment-1299096850,https://api.github.com/repos/simonw/datasette/issues/1879,1299096850,IC_kwDOBm6k_c5NbqUS,9599,simonw,2022-11-01T20:26:12Z,2022-11-01T20:26:12Z,OWNER,"The other relevant plugin here is https://datasette.io/plugins/datasette-x-forwarded-host Maybe that should be rolled into core too?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432037325,Make it easier to fix URL proxy problems, https://github.com/simonw/datasette/issues/1879#issuecomment-1299090678,https://api.github.com/repos/simonw/datasette/issues/1879,1299090678,IC_kwDOBm6k_c5Nboz2,9599,simonw,2022-11-01T20:20:28Z,2022-11-01T20:20:28Z,OWNER,My first step in debugging these is to install https://datasette.io/plugins/datasette-debug-asgi - but now I'm thinking maybe something like that should be part of core.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432037325,Make it easier to fix URL proxy problems, https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433,https://api.github.com/repos/simonw/datasette/issues/1862,1299073433,IC_kwDOBm6k_c5NbkmZ,9599,simonw,2022-11-01T20:04:31Z,2022-11-01T20:04:31Z,OWNER,"It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document): https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1425011030,"Create a new table from one or more records, `sqlite-utils` style", https://github.com/simonw/datasette/issues/1878#issuecomment-1299071456,https://api.github.com/repos/simonw/datasette/issues/1878,1299071456,IC_kwDOBm6k_c5NbkHg,9599,simonw,2022-11-01T20:02:43Z,2022-11-01T20:02:43Z,OWNER,"Note that ""update"" is partially covered by the `replace` option to `/-/insert`, added here: - https://github.com/simonw/datasette/issues/1873#issuecomment-1298885451 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1432013704,/db/table/-/upsert API, https://github.com/simonw/datasette/issues/1873#issuecomment-1298919552,https://api.github.com/repos/simonw/datasette/issues/1873,1298919552,IC_kwDOBm6k_c5Na_CA,9599,simonw,2022-11-01T18:11:27Z,2022-11-01T18:11:27Z,OWNER,"I forgot to document `ignore` and `replace`. Also I need to add tests that cover: - Forgetting to include a primary key on a non-autoincrement table - Compound primary keys - Rowid only tables with and without rowid specified I think my validation logic here will get caught out by the fact that `rowid` does not show up as a valid column name: https://github.com/simonw/datasette/blob/9bec7c38eb93cde5afb16df9bdd96aea2a5b0459/datasette/views/table.py#L1151-L1160 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1428630253,Ensure insert API has good tests for rowid and compound primark key tables, https://github.com/simonw/datasette/issues/1873#issuecomment-1298905135,https://api.github.com/repos/simonw/datasette/issues/1873,1298905135,IC_kwDOBm6k_c5Na7gv,9599,simonw,2022-11-01T17:59:59Z,2022-11-01T17:59:59Z,OWNER,"It's a bit surprising that you can send `""ignore"": true, ""return_rows"": true` and the returned `""inserted""` key will list rows that were NOT inserted (since they were ignored). Three options: 1. Ignore that and document it 2. Fix it so `""inserted""` only returns rows that were actually inserted (bit tricky) 3. Change the name of `""inserted""` to something else I'm picking 3 - I'm going to change it to be called `""rows""` instead.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1428630253,Ensure insert API has good tests for rowid and compound primark key tables, https://github.com/simonw/datasette/issues/1873#issuecomment-1298885451,https://api.github.com/repos/simonw/datasette/issues/1873,1298885451,IC_kwDOBm6k_c5Na2tL,9599,simonw,2022-11-01T17:42:20Z,2022-11-01T17:42:20Z,OWNER,"Design decision: ```json { ""rows"": [{""id"": 1, ""title"": ""The title""}], ""ignore"": true } ``` Or `""replace"": true`.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1428630253,Ensure insert API has good tests for rowid and compound primark key tables, https://github.com/simonw/sqlite-utils/issues/506#issuecomment-1298879701,https://api.github.com/repos/simonw/sqlite-utils/issues/506,1298879701,IC_kwDOCGYnMM5Na1TV,9599,simonw,2022-11-01T17:37:13Z,2022-11-01T17:37:13Z,OWNER,"The question I was originally trying to answer here was this: how many rows were actually inserted by that call to `.insert_all()`? I don't know that `.rowcount` would ever be useful here, since the ""correct"" answer depends on other factors - had I determined to ignore or replace records with a primary key that matches an existing record for example? So I think if people need `rowcount` they can get it by using a `cursor` directly.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1429029604,Make `cursor.rowcount` accessible (wontfix), https://github.com/simonw/sqlite-utils/issues/506#issuecomment-1298877872,https://api.github.com/repos/simonw/sqlite-utils/issues/506,1298877872,IC_kwDOCGYnMM5Na02w,9599,simonw,2022-11-01T17:35:30Z,2022-11-01T17:35:30Z,OWNER,"This may not make sense. First, `.last_rowid` is a property on table - but that doesn't make sense for `rowcount` since it should clearly be a property on the database itself (you can run a query directly using `db.execute()` without going through a `Table` object). So I tried this prototype: ```diff diff --git a/docs/python-api.rst b/docs/python-api.rst index 206e5e6..78d3a8d 100644 --- a/docs/python-api.rst +++ b/docs/python-api.rst @@ -186,6 +186,15 @@ The ``db.query(sql)`` function executes a SQL query and returns an iterator over # {'name': 'Cleo'} # {'name': 'Pancakes'} +After executing a query the ``db.rowcount`` property on that database instance will reflect the number of rows affected by any insert, update or delete operations performed by that query: + +.. code-block:: python + + db = Database(memory=True) + db[""dogs""].insert_all([{""name"": ""Cleo""}, {""name"": ""Pancakes""}]) + print(db.rowcount) + # Outputs: 2 + .. _python_api_execute: db.execute(sql, params) diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py index a06f4b7..c19c2dd 100644 --- a/sqlite_utils/db.py +++ b/sqlite_utils/db.py @@ -294,6 +294,8 @@ class Database: _counts_table_name = ""_counts"" use_counts_table = False + # Number of rows inserted, updated or deleted + rowcount: Optional[int] = None def __init__( self, @@ -480,9 +482,11 @@ class Database: if self._tracer: self._tracer(sql, parameters) if parameters is not None: - return self.conn.execute(sql, parameters) + cursor = self.conn.execute(sql, parameters) else: - return self.conn.execute(sql) + cursor = self.conn.execute(sql) + self.rowcount = cursor.rowcount + return cursor def executescript(self, sql: str) -> sqlite3.Cursor: """""" ``` But this happens: ```pycon >>> from sqlite_utils import Database >>> db = Database(memory=True) >>> db[""dogs""].insert_all([{""name"": ""Cleo""}, {""name"": ""Pancakes""}]) >>> db.rowcount -1 ``` Turning on query tracing demonstrates why: ```pycon >>> db = Database(memory=True, tracer=print) PRAGMA recursive_triggers=on; None >>> db[""dogs""].insert_all([{""name"": ""Cleo""}, {""name"": ""Pancakes""}]) select name from sqlite_master where type = 'view' None select name from sqlite_master where type = 'table' None select name from sqlite_master where type = 'view' None CREATE TABLE [dogs] ( [name] TEXT ); None select name from sqlite_master where type = 'view' None INSERT INTO [dogs] ([name]) VALUES (?), (?); ['Cleo', 'Pancakes'] select name from sqlite_master where type = 'table' None select name from sqlite_master where type = 'table' None PRAGMA table_info([dogs]) None
>>> ``` The `.insert_all()` function does a bunch of other queries too, so `.rowcount` is quickly over-ridden by the same result from extra queries that it executed.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1429029604,Make `cursor.rowcount` accessible (wontfix), https://github.com/simonw/datasette/issues/1876#issuecomment-1298856054,https://api.github.com/repos/simonw/datasette/issues/1876,1298856054,IC_kwDOBm6k_c5Navh2,9599,simonw,2022-11-01T17:16:01Z,2022-11-01T17:16:01Z,OWNER,`ta.style.height = ta.scrollHeight + 'px'` is an easy way to do that.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1431786951,SQL query should wrap on SQL interrupted screen, https://github.com/simonw/datasette/issues/1876#issuecomment-1298854321,https://api.github.com/repos/simonw/datasette/issues/1876,1298854321,IC_kwDOBm6k_c5NavGx,9599,simonw,2022-11-01T17:14:33Z,2022-11-01T17:14:33Z,OWNER,"I could use a `textarea` here (would need to figure out a neat pattern to expand it to fit the query): ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1431786951,SQL query should wrap on SQL interrupted screen, https://github.com/simonw/sqlite-utils/issues/507#issuecomment-1297859539,https://api.github.com/repos/simonw/sqlite-utils/issues/507,1297859539,IC_kwDOCGYnMM5NW8PT,7908073,chapmanjacobd,2022-11-01T00:40:16Z,2022-11-01T00:40:16Z,CONTRIBUTOR,"Ideally people could fix their data if they run into this issue. If you are using filenames try [convmv](https://linux.die.net/man/1/convmv) ``` convmv --preserve-mtimes -f utf8 -t utf8 --notest -i -r . ``` maybe this script will also help: ```py import argparse, shutil from pathlib import Path import ftfy from xklb import utils from xklb.utils import log def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser() parser.add_argument(""paths"", nargs='*') parser.add_argument(""--verbose"", ""-v"", action=""count"", default=0) args = parser.parse_args() log.info(utils.dict_filter_bool(args.__dict__)) return args def rename_invalid_paths() -> None: args = parse_args() for path in args.paths: log.info(path) for p in sorted([str(p) for p in Path(path).rglob(""*"")], key=len): fixed = ftfy.fix_text(p, uncurl_quotes=False).replace(""\r\n"", ""\n"").replace(""\r"", ""\n"").replace(""\n"", """") if p != fixed: try: shutil.move(p, fixed) except FileNotFoundError: log.warning(""FileNotFound. %s"", p) else: log.info(fixed) if __name__ == ""__main__"": rename_invalid_paths() ```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",1430325103,conn.execute: UnicodeEncodeError: 'utf-8' codec can't encode character,