{"id": 465815372, "node_id": "MDU6SXNzdWU0NjU4MTUzNzI=", "number": 37, "title": "Experiment with type hints", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2019-07-09T14:30:34Z", "updated_at": "2021-08-18T21:48:57Z", "closed_at": "2021-08-18T21:48:57Z", "author_association": "OWNER", "pull_request": null, "body": "Since it's designed to be used in Jupyter or for rapid prototyping in an IDE (and it's still pretty small) `sqlite-utils` feels like a great candidate for me to finally try out Python type hints.\r\n\r\nhttps://veekaybee.github.io/2019/07/08/python-type-hints/ is good.\r\n\r\nIt suggests the mypy docs for getting started: https://mypy.readthedocs.io/en/latest/existing_code.html plus this tutorial: https://pymbook.readthedocs.io/en/latest/typehinting.html", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/37/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 831751367, "node_id": "MDU6SXNzdWU4MzE3NTEzNjc=", "number": 246, "title": "Escaping FTS search strings", "user": {"value": 16001974, "label": "DeNeutoy"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2021-03-15T12:15:09Z", "updated_at": "2021-08-18T18:57:13Z", "closed_at": "2021-08-18T18:43:12Z", "author_association": "CONTRIBUTOR", "pull_request": null, "body": "\r\nThanks for the excellent library, it's very nice to use!\r\n\r\nI've been building some in memory search functionality for a data annotation tool i'm making, and I got tripped up a little bit with escaping the full text search queries. First I tried using `db.quote(q)`, which doesn't work, because sqlite FTS has it's own (separate)[ query syntax](https://www2.sqlite.org/fts5.html#full_text_query_syntax). You can see this happening here also:\r\n\r\nhttp://search-24ways.herokuapp.com/24ways-f8f455f/articles?_search=acces%2A\r\n\r\nI got around this by aggressively escaping quotes inside the query string like this:\r\n\r\n```python\r\n        quoted = q.replace('\"', '\"\"')\r\n        quoted = f'\"{quoted}\"'\r\n        print(quoted)\r\n        results = db[\"data\"].search(quoted, columns=[\"id\"])\r\n        return [x[\"id\"] for x in results]\r\n\r\n```\r\n\r\nThis works in the sense it doesn't crash, but it also removes access to the search query syntax. Given the well specified definition, it might be possible for sqlite-utils to provide a `db.quote_query(q)` which would intelligently escape a query whilst leaving the syntax intact. This would be very nice!\r\n\r\n\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/246/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 913135723, "node_id": "MDU6SXNzdWU5MTMxMzU3MjM=", "number": 266, "title": "Add some types, enforce with mypy", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 3, "created_at": "2021-06-07T06:05:56Z", "updated_at": "2021-08-18T22:25:38Z", "closed_at": "2021-08-18T22:25:38Z", "author_association": "OWNER", "pull_request": null, "body": "A good starting point would be adding type information to the members of these named tuples and the introspection methods that return them:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/9dff7a38831d471b1dff16d40d89eb5c3b4e84d6/sqlite_utils/db.py#L51-L75", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/266/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 931752773, "node_id": "MDU6SXNzdWU5MzE3NTI3NzM=", "number": 294, "title": "Add a `sqlite-utils memory` example to the README", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2021-06-28T16:35:59Z", "updated_at": "2021-08-18T21:40:03Z", "closed_at": "2021-08-18T21:40:03Z", "author_association": "OWNER", "pull_request": null, "body": "", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/294/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 934123448, "node_id": "MDU6SXNzdWU5MzQxMjM0NDg=", "number": 295, "title": "Insert with --tsv and --no-headers give error about --nl arguments", "user": {"value": 7288187, "label": "davidscotson"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-06-30T21:01:01Z", "updated_at": "2021-08-18T20:19:04Z", "closed_at": "2021-08-18T20:18:57Z", "author_association": "NONE", "pull_request": null, "body": "Not quite sure if this is a bug, or just an assumption I made but I thought `--tsv` and `--no-headers` would work together when inserting from a file, and currently they seem not to (sqlite-utils, version 3.12, installed on Mac OS X via brew)\r\n\r\nInstead it says:\r\n\r\n`Error: Use just one of --nl, --csv or --tsv`\r\n\r\nAs if it has interpreted the --no-headers as --nl.\r\n\r\nThe --help does specifically say CSV:\r\n `--no-headers              CSV file has no header row`\r\n \r\nAnd this heading in the documentation also only refers to CSV, but the text does mention TSV in passing, and I'd generally expect them to behave the same in most cases.\r\nhttps://sqlite-utils.datasette.io/en/stable/cli.html#csv-files-without-a-header-row", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/295/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 944326512, "node_id": "MDU6SXNzdWU5NDQzMjY1MTI=", "number": 296, "title": "`table.search(..., quote=True)` parameter and `sqlite-utils search --quote` option", "user": {"value": 32427188, "label": "deafmute1"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 6, "created_at": "2021-07-14T11:26:47Z", "updated_at": "2021-08-18T20:13:12Z", "closed_at": "2021-08-18T20:10:48Z", "author_association": "NONE", "pull_request": null, "body": "Hi,\r\nRecently got this error:\r\n```\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/__init__.py\", line 38, in <module>\r\n    start(\"/home/ethan/git/music-metadata-indexer/sample\", \"/home/ethan/git/music-metadata-indexer/test.db\")\r\n  File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/__init__.py\", line 23, in start\r\n    scanner.build_database()\r\n  File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/scan.py\", line 79, in build_database\r\n    _import_song(self.db, Path(dirpath).joinpath(f), self.logger) \r\n  File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/scan.py\", line 23, in _import_song\r\n    db.add_song(filepath)\r\n  File \"/home/ethan/git/music-metadata-indexer/src/mmindexer/index.py\", line 166, in add_song\r\n    for match in self.search(\"albums\", album): \r\n  File \"/home/ethan/git/music-metadata-indexer/env/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1625, in search\r\n    cursor = self.db.execute(\r\n  File \"/home/ethan/git/music-metadata-indexer/env/lib/python3.9/site-packages/sqlite_utils/db.py\", line 243, in execute\r\n    return self.conn.execute(sql, parameters)\r\nsqlite3.OperationalError: fts5: syntax error near \".\" \r\n```\r\nSo, the error seems to suggest there was a \".\" character somewhere in the SQL command that was causing the error. I did a little digging and found this in the docs: https://www.sqlite.org/fts5.html#fts5_strings. \".\" is one of the many prohibited characters.\r\n\r\nMy solution was to just strip these out of the query using this line\r\n`query = query.translate({e: None for e in itertools.chain(range(0,26), range(27, 48), range(58,65), range(91,95), [96], range(123,128))})`\r\n\r\nPerhaps this could be included into the `table.search()` function?\r\n\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/296/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 965210966, "node_id": "MDU6SXNzdWU5NjUyMTA5NjY=", "number": 314, "title": "Type signatures for `.create_table()` and `.create_table_sql()` and `.create()` and `Table.__init__`", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-08-10T18:03:59Z", "updated_at": "2021-08-18T22:25:21Z", "closed_at": "2021-08-18T22:25:21Z", "author_association": "OWNER", "pull_request": null, "body": "> Adding type signatures to `create_table()` and `.create_table_sql()` is a bit too involved, I'll do that in a separate issue.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/sqlite-utils/issues/312#issuecomment-896200682_", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/314/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 972827346, "node_id": "MDU6SXNzdWU5NzI4MjczNDY=", "number": 317, "title": "Link to a better example on docs index", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-08-17T15:43:40Z", "updated_at": "2021-08-18T18:31:43Z", "closed_at": "2021-08-18T18:31:43Z", "author_association": "OWNER", "pull_request": null, "body": "https://github.com/simonw/sqlite-utils/blob/7a19822ac9ee24be2fbb4c2326a0bf2f3d2d9c4d/docs/index.rst#L39 Is a very old example", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/317/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 972918533, "node_id": "MDU6SXNzdWU5NzI5MTg1MzM=", "number": 1438, "title": "Query page .csv and .json links are not correctly URL-encoded on Vercel under unknown specific conditions", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 7, "created_at": "2021-08-17T17:35:36Z", "updated_at": "2021-08-18T00:22:23Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "> Confirmed: https://thesession.vercel.app/thesession?sql=select+*+from+tunes+where+name+like+%22%25wise+maid%25%22%0D%0A is a page where the URL correctly encoded `%` as `%25` - but then in the HTML on that page that links to the CSV and JSON versions we get this:\r\n>\r\n> ```html\r\n> <p class=\"export-links\">This data as\r\n>   <a href=\"/thesession.json?sql=select * from tunes where name like &#34;%wise maid%&#34;\">json</a>,\r\n>   <a href=\"/thesession.csv?sql=select * from tunes where name like &#34;%wise maid%&#34;&amp;_size=max\">CSV</a>\r\n> </p>\r\n> ```\r\nThose CSV and JSON links are incorrect.\r\n\r\n_Originally posted by @simonw in https://github.com/simonw/datasette-publish-vercel/issues/48#issuecomment-900497579_", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/1438/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 974067156, "node_id": "MDU6SXNzdWU5NzQwNjcxNTY=", "number": 318, "title": "Research: handle gzipped CSV directly", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-08-18T21:23:04Z", "updated_at": "2021-08-18T21:25:30Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "Would it be worthwhile for the `sqlite-utils` command-line tool to grow features to efficiently directly interact with gzipped CSV data?\r\n\r\nMaybe add `--gz` options to both `insert` and to the various commands that output query results.", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/318/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 832687563, "node_id": "MDExOlB1bGxSZXF1ZXN0NTkzODA1ODA0", "number": 247, "title": "FTS quote functionality from datasette", "user": {"value": 16001974, "label": "DeNeutoy"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-03-16T11:17:34Z", "updated_at": "2021-08-18T18:43:12Z", "closed_at": "2021-08-18T18:43:12Z", "author_association": "CONTRIBUTOR", "pull_request": "simonw/sqlite-utils/pulls/247", "body": "Addresses #246 - this is a bit of a kludge because it doesn't actually *validate* the FTS string, just makes sure that it will not crash when executed, but I figured that building a query parser is a bit out of the scope of sqlite-utils and if you actually want to use the query language, you probably need to parse that yourself.\r\n", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/247/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}