{"id": 300580221, "node_id": "MDExOlB1bGxSZXF1ZXN0MzAwNTgwMjIx", "number": 8, "state": "closed", "locked": 0, "title": "Use less RAM", "user": {"value": 9599, "label": "simonw"}, "body": "Closes #7", "created_at": "2019-07-24T06:35:01Z", "updated_at": "2019-07-24T06:35:52Z", "closed_at": "2019-07-24T06:35:52Z", "merged_at": "2019-07-24T06:35:52Z", "merge_commit_sha": "c8392df78ee3e1643d18b747a4abf585d84d5d88", "assignee": null, "milestone": null, "draft": 0, "head": "6261500b01274a739176480774e82b31f2926e7f", "base": "5d7e14d40d5a4cfd133ca5faa442312f607784c5", "author_association": "MEMBER", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "url": "https://github.com/dogsheep/healthkit-to-sqlite/pull/8", "merged_by": null, "auto_merge": null}
{"id": 521054612, "node_id": "MDExOlB1bGxSZXF1ZXN0NTIxMDU0NjEy", "number": 13, "state": "open", "locked": 0, "title": "SQLite does not have case sensitive columns", "user": {"value": 1689944, "label": "tomaskrehlik"}, "body": "This solves a weird issue when there is record with metadata key\r\nthat is only different in letter cases.\r\n\r\nSee the test for details.", "created_at": "2020-11-14T20:12:32Z", "updated_at": "2021-08-24T13:28:26Z", "closed_at": null, "merged_at": null, "merge_commit_sha": "38856acbc724ffdb8beb9e9f4ef0dbfa8ff51ad1", "assignee": null, "milestone": null, "draft": 0, "head": "3e1b2945bc7c31be59e89c5fed86a5d2a59ebd5a", "base": "71e36e1cf034b96de2a8e6652265d782d3fdf63b", "author_association": "FIRST_TIME_CONTRIBUTOR", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "url": "https://github.com/dogsheep/healthkit-to-sqlite/pull/13", "merged_by": null, "auto_merge": null}
{"id": 561512503, "node_id": "MDExOlB1bGxSZXF1ZXN0NTYxNTEyNTAz", "number": 15, "state": "open", "locked": 0, "title": "added try / except to write_records ", "user": {"value": 9857779, "label": "ryancheley"}, "body": "to keep the data write from failing if it came across an error during processing. In particular when trying to convert my HealthKit zip file (and that of my wife's) it would consistently error out with the following:\r\n\r\n```\r\ndb.py 1709 insert_chunk\r\nresult = self.db.execute(query, params)\r\n\r\ndb.py 226 execute\r\nreturn self.conn.execute(sql, parameters)\r\n\r\nsqlite3.OperationalError:\r\ntoo many SQL variables\r\n\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------------------\r\ndb.py 1709 insert_chunk\r\nresult = self.db.execute(query, params)\r\n\r\ndb.py 226 execute\r\nreturn self.conn.execute(sql, parameters)\r\n\r\nsqlite3.OperationalError:\r\ntoo many SQL variables\r\n\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------------------\r\ndb.py 1709 insert_chunk\r\nresult = self.db.execute(query, params)\r\n\r\ndb.py 226 execute\r\nreturn self.conn.execute(sql, parameters)\r\n\r\nsqlite3.OperationalError:\r\ntable rBodyMass has no column named metadata_HKWasUserEntered\r\n\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------------------\r\nhealthkit-to-sqlite 8 <module>\r\nsys.exit(cli())\r\n\r\ncore.py 829 __call__\r\nreturn self.main(*args, **kwargs)\r\n\r\ncore.py 782 main\r\nrv = self.invoke(ctx)\r\n\r\ncore.py 1066 invoke\r\nreturn ctx.invoke(self.callback, **ctx.params)\r\n\r\ncore.py 610 invoke\r\nreturn callback(*args, **kwargs)\r\n\r\ncli.py 57 cli\r\nconvert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)\r\n\r\nutils.py 42 convert_xml_to_sqlite\r\nwrite_records(records, db)\r\n\r\nutils.py 143 write_records\r\ndb[table].insert_all(\r\n\r\ndb.py 1899 insert_all\r\nself.insert_chunk(\r\n\r\ndb.py 1720 insert_chunk\r\nself.insert_chunk(\r\n\r\ndb.py 1720 insert_chunk\r\nself.insert_chunk(\r\n\r\ndb.py 1714 insert_chunk\r\nresult = self.db.execute(query, params)\r\n\r\ndb.py 226 execute\r\nreturn self.conn.execute(sql, parameters)\r\n\r\nsqlite3.OperationalError:\r\ntable rBodyMass has no column named metadata_HKWasUserEntered\r\n```\r\n\r\nAdding the try / except in the `write_records` seems to fix that issue. ", "created_at": "2021-01-26T03:56:21Z", "updated_at": "2021-01-26T03:56:21Z", "closed_at": null, "merged_at": null, "merge_commit_sha": "8527278a87e448f57c7c6bd76a2d85f12d0233dd", "assignee": null, "milestone": null, "draft": 0, "head": "7f1b168c752b5af7c1f9052dfa61e26afc83d574", "base": "71e36e1cf034b96de2a8e6652265d782d3fdf63b", "author_association": "FIRST_TIME_CONTRIBUTOR", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "url": "https://github.com/dogsheep/healthkit-to-sqlite/pull/15", "merged_by": null, "auto_merge": null}
{"id": 592364255, "node_id": "MDExOlB1bGxSZXF1ZXN0NTkyMzY0MjU1", "number": 16, "state": "open", "locked": 0, "title": "Add a fallback ID, print if no ID found", "user": {"value": 1234956, "label": "n8henrie"}, "body": "Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/14\n", "created_at": "2021-03-13T13:38:29Z", "updated_at": "2021-03-13T14:44:04Z", "closed_at": null, "merged_at": null, "merge_commit_sha": "16ab307b2138891f226a66e4954c5470de753a0f", "assignee": null, "milestone": null, "draft": 0, "head": "27b3d54ccfe7d861770a9d0b173f6503580fea4a", "base": "71e36e1cf034b96de2a8e6652265d782d3fdf63b", "author_association": "FIRST_TIME_CONTRIBUTOR", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "url": "https://github.com/dogsheep/healthkit-to-sqlite/pull/16", "merged_by": null, "auto_merge": null}
{"id": 596627780, "node_id": "MDExOlB1bGxSZXF1ZXN0NTk2NjI3Nzgw", "number": 18, "state": "open", "locked": 0, "title": "Add datetime parsing", "user": {"value": 1234956, "label": "n8henrie"}, "body": "Parses the datetime columns so they are subsequently properly recognized as\ndatetime.\n\nFixes https://github.com/dogsheep/healthkit-to-sqlite/issues/17\n", "created_at": "2021-03-19T14:34:22Z", "updated_at": "2021-03-19T14:34:22Z", "closed_at": null, "merged_at": null, "merge_commit_sha": "c87f4e8aa88ec277c6b5a000670c2cb42a10c03d", "assignee": null, "milestone": null, "draft": 0, "head": "e0e7a0f99f844db33964b27c29b0b8d5f160202b", "base": "71e36e1cf034b96de2a8e6652265d782d3fdf63b", "author_association": "FIRST_TIME_CONTRIBUTOR", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "url": "https://github.com/dogsheep/healthkit-to-sqlite/pull/18", "merged_by": null, "auto_merge": null}
{"id": 718734191, "node_id": "MDExOlB1bGxSZXF1ZXN0NzE4NzM0MTkx", "number": 22, "state": "open", "locked": 0, "title": "Make sure that case-insensitive column names are unique", "user": {"value": 32016596, "label": "FabianHertwig"}, "body": "This closes #21.\r\n\r\nWhen there are metadata entries with the same case insensitive string, then there is an error when trying to create a new column for that metadata entry in the database table, because a column with that case insensitive name already exists.\r\n\r\n```xml\r\n <Record type=\"HKQuantityTypeIdentifierDietaryCholesterol\" sourceName=\"MyFitnessPal\" sourceVersion=\"35120\" unit=\"mg\" creationDate=\"2021-07-04 20:55:27 +0200\" startDate=\"2021-07-04 20:55:00 +0200\" endDate=\"2021-07-04 20:55:00 +0200\" value=\"124\">\r\n  <MetadataEntry key=\"meal\" value=\"Dinner\"/>\r\n  <MetadataEntry key=\"Meal\" value=\"Dinner\"/>\r\n </Record>\r\n```\r\n\r\nThe code added in this PR checks if a key already exists in a record and if so adds a number at its end. The resulting column names look like the example below then. Interestingly, the column names viewed with Datasette are not case insensitive.\r\n\r\n```text\r\nstartDate, endDate, value, unit, sourceName, sourceVersion, creationDate, metadata_meal, metadata_Meal_2, metadata_Mahlzeit\r\n```\r\n", "created_at": "2021-08-24T13:13:38Z", "updated_at": "2021-08-24T13:26:20Z", "closed_at": null, "merged_at": null, "merge_commit_sha": "c757d372c10284cd6fa58d144549bc89691341c3", "assignee": null, "milestone": null, "draft": 0, "head": "b16fb556f84a0eed262a518ca7ec82a467155d23", "base": "9fe3cb17e03d6c73222b63e643638cf951567c4c", "author_association": "FIRST_TIME_CONTRIBUTOR", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "url": "https://github.com/dogsheep/healthkit-to-sqlite/pull/22", "merged_by": null, "auto_merge": null}
{"id": 1182000455, "node_id": "PR_kwDOC8tyDs5Gc-VH", "number": 23, "state": "open", "locked": 0, "title": "Include workout statistics", "user": {"value": 2129, "label": "badboy"}, "body": "Not sure when this changed (iOS 16 maybe?), but the `WorkoutStatistics` now has a whole bunch of information about workouts, e.g. for runs it contains the distance (as a `<WorkoutStatistics type=\"HKQuantityTypeIdentifierDistanceWalkingRunning ...>` element).\r\n\r\nAdding it as another column at leat allows me to pull these out (using SQLite's JSON support).\r\nI'm running with this patch on my own data now.", "created_at": "2023-01-01T17:29:56Z", "updated_at": "2023-01-01T17:29:57Z", "closed_at": null, "merged_at": null, "merge_commit_sha": "e0ad45055f363810085119f26df87a6804451056", "assignee": null, "milestone": null, "draft": 0, "head": "d5b9e3609961515cc52bcc5ef070e3b83b473339", "base": "9fe3cb17e03d6c73222b63e643638cf951567c4c", "author_association": "FIRST_TIME_CONTRIBUTOR", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "url": "https://github.com/dogsheep/healthkit-to-sqlite/pull/23", "merged_by": null, "auto_merge": null}