{"id": 743071410, "node_id": "MDExOlB1bGxSZXF1ZXN0NTIxMDU0NjEy", "number": 13, "title": "SQLite does not have case sensitive columns", "user": {"value": 1689944, "label": "tomaskrehlik"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-11-14T20:12:32Z", "updated_at": "2021-08-24T13:28:26Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/healthkit-to-sqlite/pulls/13", "body": "This solves a weird issue when there is record with metadata key\r\nthat is only different in letter cases.\r\n\r\nSee the test for details.", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/13/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 978086284, "node_id": "MDExOlB1bGxSZXF1ZXN0NzE4NzM0MTkx", "number": 22, "title": "Make sure that case-insensitive column names are unique", "user": {"value": 32016596, "label": "FabianHertwig"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-08-24T13:13:38Z", "updated_at": "2021-08-24T13:26:20Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/healthkit-to-sqlite/pulls/22", "body": "This closes #21.\r\n\r\nWhen there are metadata entries with the same case insensitive string, then there is an error when trying to create a new column for that metadata entry in the database table, because a column with that case insensitive name already exists.\r\n\r\n```xml\r\n \r\n \r\n \r\n \r\n```\r\n\r\nThe code added in this PR checks if a key already exists in a record and if so adds a number at its end. The resulting column names look like the example below then. Interestingly, the column names viewed with Datasette are not case insensitive.\r\n\r\n```text\r\nstartDate, endDate, value, unit, sourceName, sourceVersion, creationDate, metadata_meal, metadata_Meal_2, metadata_Mahlzeit\r\n```\r\n", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/22/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}
{"id": 977128935, "node_id": "MDU6SXNzdWU5NzcxMjg5MzU=", "number": 21, "title": "Duplicate Column", "user": {"value": 32016596, "label": "FabianHertwig"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-08-23T15:00:44Z", "updated_at": "2021-08-23T17:00:59Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "Hey, thank you for this repo!\r\n\r\nWhen I try to convert my export, I get a multiple column error. Here is the stack trace:\r\n\r\n```sh\r\n(.venv) (base) computer:bodyweight_app user$ healthkit-to-sqlite ./data/Health_export.zip ./data/healthkit.db\r\nImporting from HealthKit [###############################-----] 87% 00:00:22\r\nTraceback (most recent call last):\r\n File \"/MyProject/.venv/bin/healthkit-to-sqlite\", line 10, in \r\n sys.exit(cli())\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/cli.py\", line 57, in cli\r\n convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py\", line 41, in convert_xml_to_sqlite\r\n write_records(records, db)\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/healthkit_to_sqlite/utils.py\", line 146, in write_records\r\n batch_size=50,\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py\", line 2579, in insert_all\r\n extracts=extracts,\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py\", line 1246, in create\r\n extracts=extracts,\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py\", line 767, in create_table\r\n self.execute(sql)\r\n File \"/MyProject/.venv/lib/python3.7/site-packages/sqlite_utils/db.py\", line 421, in execute\r\n return self.conn.execute(sql)\r\nsqlite3.OperationalError: duplicate column name: metadata_Meal\r\n```", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/21/reactions\", \"total_count\": 5, \"+1\": 5, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 470856782, "node_id": "MDU6SXNzdWU0NzA4NTY3ODI=", "number": 6, "title": "Break up records into different tables for each type", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-22T01:54:59Z", "updated_at": "2019-07-22T03:28:55Z", "closed_at": "2019-07-22T03:28:50Z", "author_association": "MEMBER", "pull_request": null, "body": "I don't think there's much benefit to having all of the different record types stored in the same enormous table. Here's what I get when I use `_facet=type`:\r\n\r\n![\"hello2__records__2_672_233_rows\"](\"https://user-images.githubusercontent.com/9599/61601118-e2f54d00-abe8-11e9-8bf6-3df2ef969112.png\")
\r\n\r\nI'm going to try splitting these up into separate tables - so `HKQuantityTypeIdentifierBodyMassIndex` becomes a table called `rBodyMassIndex` - and see if that's nicer to work with.", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/6/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 470637152, "node_id": "MDU6SXNzdWU0NzA2MzcxNTI=", "number": 2, "title": "Import workouts", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-20T05:20:21Z", "updated_at": "2019-07-20T06:21:41Z", "closed_at": "2019-07-20T06:21:41Z", "author_association": "MEMBER", "pull_request": null, "body": "From #1", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 470640505, "node_id": "MDU6SXNzdWU0NzA2NDA1MDU=", "number": 4, "title": "Import Records", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-20T06:11:20Z", "updated_at": "2019-07-20T06:21:41Z", "closed_at": "2019-07-20T06:21:41Z", "author_association": "MEMBER", "pull_request": null, "body": "From #1:\r\n```python\r\n 'Record': {'attr_counts': {'creationDate': 2672233,\r\n 'device': 2665111,\r\n 'endDate': 2672233,\r\n 'sourceName': 2672233,\r\n 'sourceVersion': 2671779,\r\n 'startDate': 2672233,\r\n 'type': 2672233,\r\n 'unit': 2650012,\r\n 'value': 2672232},\r\n 'child_counts': {'HeartRateVariabilityMetadataList': 2318,\r\n 'MetadataEntry': 287974},\r\n 'count': 2672233,\r\n 'parent_counts': {'Correlation': 2, 'HealthData': 2672231}},\r\n```", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/4/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 470637068, "node_id": "MDU6SXNzdWU0NzA2MzcwNjg=", "number": 1, "title": "Use XML Analyser to figure out the structure of the export XML", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-07-20T05:19:02Z", "updated_at": "2019-07-20T05:20:09Z", "closed_at": "2019-07-20T05:20:09Z", "author_association": "MEMBER", "pull_request": null, "body": "https://github.com/simonw/xml_analyser", "repo": {"value": 197882382, "label": "healthkit-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/1/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}