{"id": 560204306, "node_id": "MDExOlB1bGxSZXF1ZXN0NTYwMjA0MzA2", "number": 224, "state": "closed", "locked": 0, "title": "Add fts offset docs.", "user": {"value": 37962604, "label": "polyrand"}, "body": "The limit can be passed as a string to the query builder to have an offset. I have tested it using the shorthand `limit=f\"15, 30\"`, the standard syntax should work too.", "created_at": "2021-01-22T20:50:58Z", "updated_at": "2021-02-14T19:31:06Z", "closed_at": "2021-02-14T19:31:06Z", "merged_at": null, "merge_commit_sha": "4d6ff040770119fb2c1bcbc97678d9deca752f2f", "assignee": null, "milestone": null, "draft": 0, "head": "341f50d2d95ba1d69ad64ba8c0ec0ffa9a68d063", "base": "36dc7e3909a44878681c266b90f9be76ac749f2d", "author_association": "NONE", "repo": {"value": 140912432, "label": "sqlite-utils"}, "url": "https://github.com/simonw/sqlite-utils/pull/224", "merged_by": null, "auto_merge": null}
{"id": 564215011, "node_id": "MDExOlB1bGxSZXF1ZXN0NTY0MjE1MDEx", "number": 225, "state": "closed", "locked": 0, "title": "fix for problem in Table.insert_all on search for columns per chunk of rows", "user": {"value": 261237, "label": "nieuwenhoven"}, "body": "Hi,\r\n\r\nI ran into a problem when trying to create a database from my Apple Healthkit data using [healthkit-to-sqlite](https://github.com/dogsheep/healthkit-to-sqlite). The program crashed because of an invalid insert statement that was generated for table `rDistanceCycling`. \r\n\r\nThe actual problem turned out to be in [sqlite-utils](https://github.com/simonw/sqlite-utils). `Table.insert_all` processes the data to be inserted in chunks of rows and checks for every chunk which columns are used, and it will collect all column names in the variable `all_columns`.  The collection of columns is done using a nested list comprehension that is not completely correct. \r\n\r\nI'm using a Windows machine and had to make a few adjustments to the tests in order to be able to run them because they had a posix dependency.\r\n\r\nThanks, kind regards,\r\n\r\nFrans\r\n\r\n```\r\n# this is a (condensed) chunk of data from my Apple healthkit export that caused the problem.\r\n# the 3 last items in the chunk have additional keys: metadata_HKMetadataKeySyncVersion and metadata_HKMetadataKeySyncIdentifier\r\n\r\nchunk = [{'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '7.0.1',\r\n          'device': '<<HKDevice: 0x281cf6c70>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:7.0.1>',\r\n          'unit': 'km', 'creationDate': '2020-10-10 12:29:09 +0100', 'startDate': '2020-10-10 12:29:06 +0100',\r\n          'endDate': '2020-10-10 12:29:07 +0100', 'value': '0.00518016'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '7.0.1',\r\n          'device': '<<HKDevice: 0x281cf6c70>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:7.0.1>',\r\n          'unit': 'km', 'creationDate': '2020-10-10 12:29:10 +0100', 'startDate': '2020-10-10 12:29:07 +0100',\r\n          'endDate': '2020-10-10 12:29:08 +0100', 'value': '0.00544049'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '6.2.6',\r\n          'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>',\r\n          'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:40:50 +0100',\r\n          'endDate': '2020-07-15 16:42:49 +0100', 'value': '0.952092', 'metadata_HKMetadataKeySyncVersion': '1',\r\n          'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520450.99823:616520569.99360:119'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '6.2.6',\r\n          'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>',\r\n          'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:42:49 +0100',\r\n          'endDate': '2020-07-15 16:44:51 +0100', 'value': '0.848983', 'metadata_HKMetadataKeySyncVersion': '1',\r\n          'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520569.99360:616520691.98826:119'},\r\n         {'sourceName': 'Apple\u00c2\\xa0Watch van Frans', 'sourceVersion': '6.2.6',\r\n          'device': '<<HKDevice: 0x281cf83e0>, name:Apple Watch, manufacturer:Apple Inc., model:Watch, hardware:Watch3,4, software:6.2.6>',\r\n          'unit': 'km', 'creationDate': '2020-10-14 05:54:12 +0100', 'startDate': '2020-07-15 16:44:51 +0100',\r\n          'endDate': '2020-07-15 16:46:50 +0100', 'value': '0.834403', 'metadata_HKMetadataKeySyncVersion': '1',\r\n          'metadata_HKMetadataKeySyncIdentifier': '3:674DBCDB-3FE8-40D1-9FC1-E54A2B413805:616520691.98826:616520810.98305:119'}]\r\n\r\n\r\n\r\ndef all_columns_old():\r\n    all_columns = [col for col in chunk[0]]\r\n    all_columns += [column for record in chunk\r\n                           for column in record if column not in all_columns]\r\n    return all_columns\r\n\r\n\r\ndef all_columns_new():\r\n    all_columns = [col for col in chunk[0]]\r\n    for record in chunk:\r\n        all_columns += [column for column in record if column not in all_columns]\r\n    return all_columns\r\n\r\n\r\n\r\nif __name__ == '__main__':\r\n    from pprint import pprint\r\n\r\n    print('problem: ')\r\n    pprint(all_columns_old())\r\n    print('\\nfix: ')\r\n    pprint(all_columns_new())\r\n\r\n```\r\n", "created_at": "2021-01-29T20:16:07Z", "updated_at": "2021-02-14T21:04:13Z", "closed_at": "2021-02-14T21:04:13Z", "merged_at": null, "merge_commit_sha": "1cba965a1ddc2bd77db3bc3912aa7e8467e2fa2f", "assignee": null, "milestone": null, "draft": 0, "head": "929ea7551135df0cc2ac9d67f4fbbecf701a11f6", "base": "36dc7e3909a44878681c266b90f9be76ac749f2d", "author_association": "NONE", "repo": {"value": 140912432, "label": "sqlite-utils"}, "url": "https://github.com/simonw/sqlite-utils/pull/225", "merged_by": null, "auto_merge": null}
{"id": 573147168, "node_id": "MDExOlB1bGxSZXF1ZXN0NTczMTQ3MTY4", "number": 233, "state": "closed", "locked": 0, "title": "Run tests against Ubuntu, macOS and Windows", "user": {"value": 9599, "label": "simonw"}, "body": "Refs #232", "created_at": "2021-02-14T20:11:02Z", "updated_at": "2021-02-14T20:39:54Z", "closed_at": "2021-02-14T20:39:54Z", "merged_at": "2021-02-14T20:39:54Z", "merge_commit_sha": "f51a1f6c3cb2929bcf79cb4efe3b2a9886d9c25c", "assignee": null, "milestone": null, "draft": 0, "head": "deb01a8913441e7fedb53c52a3a9fdff274ddbd2", "base": "320f3ac33a83b32f89559ef0c162b7eca428a278", "author_association": "OWNER", "repo": {"value": 140912432, "label": "sqlite-utils"}, "url": "https://github.com/simonw/sqlite-utils/pull/233", "merged_by": null, "auto_merge": null}