issue_comments

Hmm... is it even possible to fix this within sqlite-utils? Maybe this is a SQLite implementation detail.

It's possible to update the schema itself - that's how we add missing foreign key constraints - but it seems a little bit of overkill to use the writable schema when adding a new column JUST to get prettier indentation:

https://github.com/simonw/sqlite-utils/blob/c88f0a4d4617ff832aa728aac1da3500548c3137/sqlite_utils/db.py#L316-L326

Here's the implementation in create_table():

https://github.com/simonw/sqlite-utils/blob/c88f0a4d4617ff832aa728aac1da3500548c3137/sqlite_utils/db.py#L235-L249

And here's where add_column() does its thing:

https://github.com/simonw/sqlite-utils/blob/c88f0a4d4617ff832aa728aac1da3500548c3137/sqlite_utils/db.py#L547-L552

Oh yes well spotted thank you 😁

I agree that the strictness would be nice as it could help to avoid different middleware altering the scope in incompatible ways. However I do also agree that it's likely for not all implementations to follow 🤔

Right now the spec does say “copy the scope, rather than mutate it” https://asgi.readthedocs.io/en/latest/specs/main.html#middleware

I wouldn’t be surprised if that there’s room for discussion on evolving the exact language there.

There’s obvs a nice element to the strictness there, tho practically I’m not sure it’s something that implementations will follow, and its not something that Starlette chooses to abide by.

Some examples:

https://github.com/dogsheep/healthkit-to-sqlite/blob/d016e70c31cf84ba0f5ec3102546db54a51aaffb/tests/export.xml#L4-L13

The asgi spec doesn't explicitly specify (at least as far as I can tell) whether the scope is immutable/mutable https://asgi.readthedocs.io/en/latest/specs/lifespan.html#scope . @simonw using a header for this would be a nice approach. It would also potentially increase the portability of any middleware/plugins/clients across different applications/frameworks as it's not tied directly to an asgi implementation

python 'Workout': {'attr_counts': {'creationDate': 663, 'device': 230, 'duration': 663, 'durationUnit': 663, 'endDate': 663, 'sourceName': 663, 'sourceVersion': 663, 'startDate': 663, 'totalDistance': 663, 'totalDistanceUnit': 663, 'totalEnergyBurned': 663, 'totalEnergyBurnedUnit': 663, 'workoutActivityType': 663}, 'child_counts': {'MetadataEntry': 1928, 'WorkoutEvent': 2094, 'WorkoutRoute': 340}, 'count': 663, 'parent_counts': {'HealthData': 663}}, 'WorkoutEvent': {'attr_counts': {'date': 2094, 'duration': 837, 'durationUnit': 837, 'type': 2094}, 'child_counts': {}, 'count': 2094, 'parent_counts': {'Workout': 2094}}, 'WorkoutRoute': {'attr_counts': {'creationDate': 340, 'endDate': 340, 'sourceName': 340, 'sourceVersion': 340, 'startDate': 340}, 'child_counts': {'Location': 398683, 'MetadataEntry': 546}, 'count': 340, 'parent_counts': {'Workout': 340}}}

I ran xml_analyser against the XML HealthKit export.xml file and got the following results:

python { 'ActivitySummary': {'attr_counts': {'activeEnergyBurned': 980, 'activeEnergyBurnedGoal': 980, 'activeEnergyBurnedUnit': 980, 'appleExerciseTime': 980, 'appleExerciseTimeGoal': 980, 'appleStandHours': 980, 'appleStandHoursGoal': 980, 'dateComponents': 980}, 'child_counts': {}, 'count': 980, 'parent_counts': {'HealthData': 980}}, 'Correlation': {'attr_counts': {'creationDate': 1, 'endDate': 1, 'sourceName': 1, 'sourceVersion': 1, 'startDate': 1, 'type': 1}, 'child_counts': {'MetadataEntry': 1, 'Record': 2}, 'count': 1, 'parent_counts': {'HealthData': 1}}, 'ExportDate': {'attr_counts': {'value': 1}, 'child_counts': {}, 'count': 1, 'parent_counts': {'HealthData': 1}}, 'HealthData': {'attr_counts': {'locale': 1}, 'child_counts': {'ActivitySummary': 980, 'Correlation': 1, 'ExportDate': 1, 'Me': 1, 'Record': 2672231, 'Workout': 663}, 'count': 1, 'parent_counts': {}}, 'HeartRateVariabilityMetadataList': {'attr_counts': {}, 'child_counts': {'InstantaneousBeatsPerMinute': 93653}, 'count': 2318, 'parent_counts': {'Record': 2318}}, 'InstantaneousBeatsPerMinute': {'attr_counts': {'bpm': 93653, 'time': 93653}, 'child_counts': {}, 'count': 93653, 'parent_counts': {'HeartRateVariabilityMetadataList': 93653}}, 'Location': {'attr_counts': {'altitude': 398683, 'course': 398683, 'date': 398683, 'horizontalAccuracy': 398683, 'latitude': 398683, 'longitude': 398683, 'speed': 398683, 'verticalAccuracy': 398683}, 'child_counts': {}, 'count': 398683, 'parent_counts': {'WorkoutRoute': 398683}}, 'Me': {'attr_counts': {'HKCharacteristicTypeIdentifierBiologicalSex': 1, 'HKCharacteristicTypeIdentifierBloodType': 1, 'HKCharacteristicTypeIdentifierDateOfBirth': 1, 'HKCharacteristicTypeIdentifierFitzpatrickSkinType': 1}, 'child_counts': {}, 'count': 1, 'parent_counts': {'HealthData': 1}}, 'MetadataEntry': {'attr_counts': {'key': 290449, 'value': 290449}, 'child_counts': {}, 'count': 290449, 'parent_counts': {'Correlation': 1, 'Record': 287974, 'Workout': 1928, 'WorkoutRoute': 546}}, 'Record': {'attr_counts': {'creationDate': 2672233, 'device': 2665111, 'endDate': 2672233, 'sourceName': 2672233, 'sourceVersion': 2671779, 'startDate': 2672233, 'type': 2672233, 'unit': 2650012, 'value': 2672232}, 'child_counts': {'HeartRateVariabilityMetadataList': 2318, 'MetadataEntry': 287974}, 'count': 2672233, 'parent_counts': {'Correlation': 2, 'HealthData': 2672231}}, 'Workout': {'attr_counts': {'creationDate': 663, 'device': 230, 'duration': 663, 'durationUnit': 663, 'endDate': 663, 'sourceName': 663, 'sourceVersion': 663, 'startDate': 663, 'totalDistance': 663, 'totalDistanceUnit': 663, 'totalEnergyBurned': 663, 'totalEnergyBurnedUnit': 663, 'workoutActivityType': 663}, 'child_counts': {'MetadataEntry': 1928, 'WorkoutEvent': 2094, 'WorkoutRoute': 340}, 'count': 663, 'parent_counts': {'HealthData': 663}}, 'WorkoutEvent': {'attr_counts': {'date': 2094, 'duration': 837, 'durationUnit': 837, 'type': 2094}, 'child_counts': {}, 'count': 2094, 'parent_counts': {'Workout': 2094}}, 'WorkoutRoute': {'attr_counts': {'creationDate': 340, 'endDate': 340, 'sourceName': 340, 'sourceVersion': 340, 'startDate': 340}, 'child_counts': {'Location': 398683, 'MetadataEntry': 546}, 'count': 340, 'parent_counts': {'Workout': 340}}}

The most interesting bit is this:

python 'HealthData': {'attr_counts': {'locale': 1}, 'child_counts': {'ActivitySummary': 980, 'Correlation': 1, 'ExportDate': 1, 'Me': 1, 'Record': 2672231, 'Workout': 663},