{"id": 1515883470, "node_id": "I_kwDOC8tyDs5aWovO", "number": 24, "title": "DOC: xml.etree.ElementTree.ParseError due to healthkit version 12 ", "user": {"value": 6231413, "label": "mmngreco"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2023-01-01T23:00:38Z", "updated_at": "2023-03-30T10:17:31Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hi @simonw \r\n\r\nI hope you find this issue ok, the idea is provide some documentation to other users like me about how to solve this problem and save some time.\r\n\r\nFollowing the instructions from the `README.md` I've faced this error:\r\n\r\n```bash\r\n(venv) mgreco@pop-os apple-health master* (23:44|0s)\r\n$ healthkit-to-sqlite apple_health_export/export.xml healthkit.db --xml\r\nImporting from HealthKit  [------------------------------------]    0%\r\nTraceback (most recent call last):\r\n  File \"/home/mgreco/github/mmngreco/apple-health/venv/bin/healthkit-to-sqlite\", line 33, in <module>\r\n    sys.exit(load_entry_point('healthkit-to-sqlite', 'console_scripts', 'healthkit-to-sqlite')())\r\n  File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 1130, in __call__\r\n    return self.main(*args, **kwargs)\r\n  File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 1055, in main\r\n    rv = self.invoke(ctx)\r\n  File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 1404, in invoke\r\n    return ctx.invoke(self.callback, **ctx.params)\r\n  File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/site-packages/click/core.py\", line 760, in invoke\r\n    return __callback(*args, **kwargs)\r\n  File \"/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/cli.py\", line 57, in cli\r\n    convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)\r\n  File \"/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py\", line 25, in convert_xml_to_sqlite\r\n    for tag, el in find_all_tags(\r\n  File \"/home/mgreco/github/mmngreco/apple-health/.deps/healthkit-to-sqlite/healthkit_to_sqlite/utils.py\", line 12, in find_all_tags\r\n    for event, el in parser.read_events():\r\n  File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/xml/etree/ElementTree.py\", line 1324, in read_events\r\n    raise event\r\n  File \"/home/mgreco/github/mmngreco/apple-health/venv/lib/python3.10/xml/etree/ElementTree.py\", line 1296, in feed\r\n    self._parser.feed(data)\r\nxml.etree.ElementTree.ParseError: syntax error: line 156, column 0\r\n```\r\n\r\nSo, after debugging and searching on internet I found this useful link: https://discussions.apple.com/thread/254202523 (etresoft, the real hero). Which basically says that the xml given by the health app (healthkit version 12) has some bugs but  fortunately, they can be solved with a couple of commads:\r\n\r\n1. Uncompress the zip and move the new folder where `export.xml` is.\r\n1. Create a `patch.txt` with the following content\r\n\r\n    ```diff\r\n    --- export.xml\t2022-09-18 15:17:09.000000000 -0400\r\n    +++ export-fixed.xml\t2022-09-18 16:37:08.000000000 -0400\r\n    @@ -15,6 +15,7 @@\r\n       HKCharacteristicTypeIdentifierBiologicalSex       CDATA #REQUIRED\r\n       HKCharacteristicTypeIdentifierBloodType           CDATA #REQUIRED\r\n       HKCharacteristicTypeIdentifierFitzpatrickSkinType CDATA #REQUIRED\r\n    +  HKCharacteristicTypeIdentifierCardioFitnessMedicationsUse CDATA #IMPLIED\r\n     >\r\n     <!ELEMENT Record ((MetadataEntry|HeartRateVariabilityMetadataList)*)>\r\n     <!ATTLIST Record\r\n    @@ -39,7 +40,7 @@\r\n       startDate     CDATA #REQUIRED\r\n       endDate       CDATA #REQUIRED\r\n     >\r\n    -<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute)*)>\r\n    +<!ELEMENT Workout ((MetadataEntry|WorkoutEvent|WorkoutRoute|WorkoutStatistics)*)>\r\n     <!ATTLIST Workout\r\n       workoutActivityType   CDATA #REQUIRED\r\n       duration              CDATA #IMPLIED\r\n    @@ -63,7 +64,7 @@\r\n       duration             CDATA #IMPLIED\r\n       durationUnit         CDATA #IMPLIED\r\n     >\r\n    -<!ELEMENT WorkoutEvent EMPTY>\r\n    +<!ELEMENT WorkoutEvent (MetadataEntry?)>\r\n     <!ATTLIST WorkoutEvent\r\n       type                 CDATA #REQUIRED\r\n       date                 CDATA #REQUIRED\r\n    @@ -79,6 +80,7 @@\r\n       minimum              CDATA #IMPLIED\r\n       maximum              CDATA #IMPLIED\r\n       sum                  CDATA #IMPLIED\r\n    +  unit                 CDATA #IMPLIED\r\n     >\r\n     <!ELEMENT WorkoutRoute ((MetadataEntry|FileReference)*)>\r\n     <!ATTLIST WorkoutRoute\r\n    @@ -153,6 +155,7 @@\r\n       dateIssued       CDATA #REQUIRED\r\n       expirationDate   CDATA #REQUIRED\r\n       brand            CDATA #IMPLIED\r\n    +>\r\n     <!ELEMENT RightEye EMPTY>\r\n     <!ATTLIST RightEye\r\n       sphere           CDATA #IMPLIED\r\n    @@ -203,13 +206,6 @@\r\n       diameter         CDATA #IMPLIED\r\n       diameterUnit     CDATA #IMPLIED\r\n     >\r\n    -  device           CDATA #IMPLIED\r\n    -<!ELEMENT MetadataEntry EMPTY>\r\n    -<!ATTLIST MetadataEntry\r\n    -  key              CDATA #IMPLIED\r\n    -  value            CDATA #IMPLIED\r\n    ->\r\n    ->\r\n     ]>\r\n     <HealthData>\r\n      <ExportDate/>\r\n    ```\r\n1. Apply the path with the command: `patch < patch.txt`\r\n1. Fix endDates with the command `sed 's/startDate/endDate/2' export.xml > export-fixed.xml`\r\n1. Try again `healthkit-to-sqlite export-fixed.xml healthkit.db --xml`", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/24/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 975166271, "node_id": "MDU6SXNzdWU5NzUxNjYyNzE=", "number": 20, "title": "Add index on workout_points.date", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2021-08-20T01:08:04Z", "updated_at": "2021-08-20T01:12:48Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "Sorting that by date makes sense for seeing most recent points, and my DB has 2.5m points in so it's an expensive sort!", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/20/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 470691622, "node_id": "MDU6SXNzdWU0NzA2OTE2MjI=", "number": 5, "title": "Add progress bar", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2019-07-20T16:29:07Z", "updated_at": "2019-07-22T03:30:13Z", "closed_at": "2019-07-22T02:49:22Z", "author_association": "MEMBER", "pull_request": null, "body": "Showing a progress bar would be nice, using Click.\r\n\r\nThe easiest way to do this would probably be be to hook it up to the length of the compressed content, and update it as this code pushes more XML bytes through the parser:\r\n\r\nhttps://github.com/dogsheep/healthkit-to-sqlite/blob/d64299765064501f4efdd9a0b21dbdba9ec4287f/healthkit_to_sqlite/utils.py#L6-L10", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}