github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/507#issuecomment-1297859539 | https://api.github.com/repos/simonw/sqlite-utils/issues/507 | 1297859539 | IC_kwDOCGYnMM5NW8PT | 7908073 | 2022-11-01T00:40:16Z | 2022-11-01T00:40:16Z | CONTRIBUTOR | Ideally people could fix their data if they run into this issue. If you are using filenames try [convmv](https://linux.die.net/man/1/convmv) ``` convmv --preserve-mtimes -f utf8 -t utf8 --notest -i -r . ``` maybe this script will also help: ```py import argparse, shutil from pathlib import Path import ftfy from xklb import utils from xklb.utils import log def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser() parser.add_argument("paths", nargs='*') parser.add_argument("--verbose", "-v", action="count", default=0) args = parser.parse_args() log.info(utils.dict_filter_bool(args.__dict__)) return args def rename_invalid_paths() -> None: args = parse_args() for path in args.paths: log.info(path) for p in sorted([str(p) for p in Path(path).rglob("*")], key=len): fixed = ftfy.fix_text(p, uncurl_quotes=False).replace("\r\n", "\n").replace("\r", "\n").replace("\n", "") if p != fixed: try: shutil.move(p, fixed) except FileNotFoundError: log.warning("FileNotFound. %s", p) else: log.info(fixed) if __name__ == "__main__": rename_invalid_paths() ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1430325103 | |
https://github.com/simonw/datasette/issues/1876#issuecomment-1298854321 | https://api.github.com/repos/simonw/datasette/issues/1876 | 1298854321 | IC_kwDOBm6k_c5NavGx | 9599 | 2022-11-01T17:14:33Z | 2022-11-01T17:14:33Z | OWNER | I could use a `textarea` here (would need to figure out a neat pattern to expand it to fit the query): <img width="426" alt="image" src="https://user-images.githubusercontent.com/9599/199295041-25abe0b9-f825-43a2-ae5a-face622e08bc.png"> | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1431786951 | |
https://github.com/simonw/datasette/issues/1876#issuecomment-1298856054 | https://api.github.com/repos/simonw/datasette/issues/1876 | 1298856054 | IC_kwDOBm6k_c5Navh2 | 9599 | 2022-11-01T17:16:01Z | 2022-11-01T17:16:01Z | OWNER | `ta.style.height = ta.scrollHeight + 'px'` is an easy way to do that. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1431786951 | |
https://github.com/simonw/sqlite-utils/issues/506#issuecomment-1298877872 | https://api.github.com/repos/simonw/sqlite-utils/issues/506 | 1298877872 | IC_kwDOCGYnMM5Na02w | 9599 | 2022-11-01T17:35:30Z | 2022-11-01T17:35:30Z | OWNER | This may not make sense. First, `.last_rowid` is a property on table - but that doesn't make sense for `rowcount` since it should clearly be a property on the database itself (you can run a query directly using `db.execute()` without going through a `Table` object). So I tried this prototype: ```diff diff --git a/docs/python-api.rst b/docs/python-api.rst index 206e5e6..78d3a8d 100644 --- a/docs/python-api.rst +++ b/docs/python-api.rst @@ -186,6 +186,15 @@ The ``db.query(sql)`` function executes a SQL query and returns an iterator over # {'name': 'Cleo'} # {'name': 'Pancakes'} +After executing a query the ``db.rowcount`` property on that database instance will reflect the number of rows affected by any insert, update or delete operations performed by that query: + +.. code-block:: python + + db = Database(memory=True) + db["dogs"].insert_all([{"name": "Cleo"}, {"name": "Pancakes"}]) + print(db.rowcount) + # Outputs: 2 + .. _python_api_execute: db.execute(sql, params) diff --git a/sqlite_utils/db.py b/sqlite_utils/db.py index a06f4b7..c19c2dd 100644 --- a/sqlite_utils/db.py +++ b/sqlite_utils/db.py @@ -294,6 +294,8 @@ class Database: _counts_table_name = "_counts" use_counts_table = False + # Number of rows inserted, updated or deleted + rowcount: Optional[int] = None def __init__( self, @@ -480,9 +482,11 @@ class Database: if self._tracer: self._tracer(sql, parameters) if parameters is not None: - return self.conn.execute(sql, parameters) + cursor = self.conn.execute(sql, parameters) else: - return self.conn.execute(sql) + cursor = self.conn.execute(sql) + self.rowcount = cursor.rowcount + return cursor def executescript(self, sql: str) -> sqlite3.Cursor: """ ``` But this happens: ```pycon >>> from sqlite_utils import Database >>> db = Database(memory=True) >>> db["dogs"].insert_a… | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1429029604 | |
https://github.com/simonw/sqlite-utils/issues/506#issuecomment-1298879701 | https://api.github.com/repos/simonw/sqlite-utils/issues/506 | 1298879701 | IC_kwDOCGYnMM5Na1TV | 9599 | 2022-11-01T17:37:13Z | 2022-11-01T17:37:13Z | OWNER | The question I was originally trying to answer here was this: how many rows were actually inserted by that call to `.insert_all()`? I don't know that `.rowcount` would ever be useful here, since the "correct" answer depends on other factors - had I determined to ignore or replace records with a primary key that matches an existing record for example? So I think if people need `rowcount` they can get it by using a `cursor` directly. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1429029604 | |
https://github.com/simonw/datasette/issues/1873#issuecomment-1298885451 | https://api.github.com/repos/simonw/datasette/issues/1873 | 1298885451 | IC_kwDOBm6k_c5Na2tL | 9599 | 2022-11-01T17:42:20Z | 2022-11-01T17:42:20Z | OWNER | Design decision: ```json { "rows": [{"id": 1, "title": "The title"}], "ignore": true } ``` Or `"replace": true`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1428630253 | |
https://github.com/simonw/datasette/issues/1873#issuecomment-1298905135 | https://api.github.com/repos/simonw/datasette/issues/1873 | 1298905135 | IC_kwDOBm6k_c5Na7gv | 9599 | 2022-11-01T17:59:59Z | 2022-11-01T17:59:59Z | OWNER | It's a bit surprising that you can send `"ignore": true, "return_rows": true` and the returned `"inserted"` key will list rows that were NOT inserted (since they were ignored). Three options: 1. Ignore that and document it 2. Fix it so `"inserted"` only returns rows that were actually inserted (bit tricky) 3. Change the name of `"inserted"` to something else I'm picking 3 - I'm going to change it to be called `"rows"` instead. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1428630253 | |
https://github.com/simonw/datasette/issues/1873#issuecomment-1298919552 | https://api.github.com/repos/simonw/datasette/issues/1873 | 1298919552 | IC_kwDOBm6k_c5Na_CA | 9599 | 2022-11-01T18:11:27Z | 2022-11-01T18:11:27Z | OWNER | I forgot to document `ignore` and `replace`. Also I need to add tests that cover: - Forgetting to include a primary key on a non-autoincrement table - Compound primary keys - Rowid only tables with and without rowid specified I think my validation logic here will get caught out by the fact that `rowid` does not show up as a valid column name: https://github.com/simonw/datasette/blob/9bec7c38eb93cde5afb16df9bdd96aea2a5b0459/datasette/views/table.py#L1151-L1160 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1428630253 | |
https://github.com/simonw/datasette/issues/1878#issuecomment-1299071456 | https://api.github.com/repos/simonw/datasette/issues/1878 | 1299071456 | IC_kwDOBm6k_c5NbkHg | 9599 | 2022-11-01T20:02:43Z | 2022-11-01T20:02:43Z | OWNER | Note that "update" is partially covered by the `replace` option to `/-/insert`, added here: - https://github.com/simonw/datasette/issues/1873#issuecomment-1298885451 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1432013704 | |
https://github.com/simonw/datasette/issues/1862#issuecomment-1299073433 | https://api.github.com/repos/simonw/datasette/issues/1862 | 1299073433 | IC_kwDOBm6k_c5NbkmZ | 9599 | 2022-11-01T20:04:31Z | 2022-11-01T20:04:31Z | OWNER | It really feels like this should be accompanied by a `/db/-/create` API for creating tables. I had to add that to `sqlite-utils` eventually (initially it only supported creating by passing in an example document): https://sqlite-utils.datasette.io/en/stable/cli.html#cli-create-table | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1425011030 | |
https://github.com/simonw/datasette/issues/1879#issuecomment-1299090678 | https://api.github.com/repos/simonw/datasette/issues/1879 | 1299090678 | IC_kwDOBm6k_c5Nboz2 | 9599 | 2022-11-01T20:20:28Z | 2022-11-01T20:20:28Z | OWNER | My first step in debugging these is to install https://datasette.io/plugins/datasette-debug-asgi - but now I'm thinking maybe something like that should be part of core. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1432037325 | |
https://github.com/simonw/datasette/issues/1879#issuecomment-1299096850 | https://api.github.com/repos/simonw/datasette/issues/1879 | 1299096850 | IC_kwDOBm6k_c5NbqUS | 9599 | 2022-11-01T20:26:12Z | 2022-11-01T20:26:12Z | OWNER | The other relevant plugin here is https://datasette.io/plugins/datasette-x-forwarded-host Maybe that should be rolled into core too? | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1432037325 | |
https://github.com/simonw/datasette/issues/1879#issuecomment-1299098458 | https://api.github.com/repos/simonw/datasette/issues/1879 | 1299098458 | IC_kwDOBm6k_c5Nbqta | 9599 | 2022-11-01T20:27:40Z | 2022-11-01T20:33:52Z | OWNER | https://github.com/simonw/datasette-x-forwarded-host/blob/main/datasette_x_forwarded_host/__init__.py could happen in core controlled by: `--setting trust_forwarded_host 1` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1432037325 | |
https://github.com/simonw/datasette/issues/1879#issuecomment-1299102108 | https://api.github.com/repos/simonw/datasette/issues/1879 | 1299102108 | IC_kwDOBm6k_c5Nbrmc | 9599 | 2022-11-01T20:30:54Z | 2022-11-01T20:33:06Z | OWNER | One idea: add a `/-/debug` page (or `/-/tips` or `/-/checks`) which shows the incoming requests headers and could even detect if there's an `x-forwarded-host` header that isn't being repeated and show a tip on how to fix that. | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1432037325 | |
https://github.com/simonw/datasette/issues/1879#issuecomment-1299102755 | https://api.github.com/repos/simonw/datasette/issues/1879 | 1299102755 | IC_kwDOBm6k_c5Nbrwj | 9599 | 2022-11-01T20:31:37Z | 2022-11-01T20:31:37Z | OWNER | And some JavaScript that can spot if Datasette thinks it is being served over HTTP when it's actually being served over HTTPS. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1432037325 | |
https://github.com/simonw/datasette/issues/1871#issuecomment-1299349741 | https://api.github.com/repos/simonw/datasette/issues/1871 | 1299349741 | IC_kwDOBm6k_c5NcoDt | 9599 | 2022-11-01T23:22:55Z | 2022-11-01T23:22:55Z | OWNER | It's weird that the API explorer only lets you explore POST APIs. It should probably also let you explore GET APIs, or be renamed. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1427293909 |