issues

Table actions

6 rows where repo = 197882382, state = "open" and type = "pull" sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: user, comments, created_at (date), updated_at (date)

type 1

  • pull · 6 ✖

state 1

  • open · 6 ✖

repo 1

  • healthkit-to-sqlite · 6 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1515717718 PR_kwDOC8tyDs5Gc-VH 23 Include workout statistics badboy 2129 open 0     0 2023-01-01T17:29:57Z 2023-01-01T17:29:57Z   FIRST_TIME_CONTRIBUTOR dogsheep/healthkit-to-sqlite/pulls/23

Not sure when this changed (iOS 16 maybe?), but the WorkoutStatistics now has a whole bunch of information about workouts, e.g. for runs it contains the distance (as a <WorkoutStatistics type="HKQuantityTypeIdentifierDistanceWalkingRunning ...> element).

Adding it as another column at leat allows me to pull these out (using SQLite's JSON support). I'm running with this patch on my own data now.

 healthkit-to-sqlite 197882382 pull     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/23/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
743071410 MDExOlB1bGxSZXF1ZXN0NTIxMDU0NjEy 13 SQLite does not have case sensitive columns tomaskrehlik 1689944 open 0     1 2020-11-14T20:12:32Z 2021-08-24T13:28:26Z   FIRST_TIME_CONTRIBUTOR dogsheep/healthkit-to-sqlite/pulls/13

This solves a weird issue when there is record with metadata key that is only different in letter cases.

See the test for details.

 healthkit-to-sqlite 197882382 pull     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/13/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
978086284 MDExOlB1bGxSZXF1ZXN0NzE4NzM0MTkx 22 Make sure that case-insensitive column names are unique FabianHertwig 32016596 open 0     1 2021-08-24T13:13:38Z 2021-08-24T13:26:20Z   FIRST_TIME_CONTRIBUTOR dogsheep/healthkit-to-sqlite/pulls/22

This closes #21.

When there are metadata entries with the same case insensitive string, then there is an error when trying to create a new column for that metadata entry in the database table, because a column with that case insensitive name already exists.

xml <Record type="HKQuantityTypeIdentifierDietaryCholesterol" sourceName="MyFitnessPal" sourceVersion="35120" unit="mg" creationDate="2021-07-04 20:55:27 +0200" startDate="2021-07-04 20:55:00 +0200" endDate="2021-07-04 20:55:00 +0200" value="124"> <MetadataEntry key="meal" value="Dinner"/> <MetadataEntry key="Meal" value="Dinner"/> </Record>

The code added in this PR checks if a key already exists in a record and if so adds a number at its end. The resulting column names look like the example below then. Interestingly, the column names viewed with Datasette are not case insensitive.

text startDate, endDate, value, unit, sourceName, sourceVersion, creationDate, metadata_meal, metadata_Meal_2, metadata_Mahlzeit

 healthkit-to-sqlite 197882382 pull     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/22/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
836064851 MDExOlB1bGxSZXF1ZXN0NTk2NjI3Nzgw 18 Add datetime parsing n8henrie 1234956 open 0     0 2021-03-19T14:34:22Z 2021-03-19T14:34:22Z   FIRST_TIME_CONTRIBUTOR dogsheep/healthkit-to-sqlite/pulls/18

Parses the datetime columns so they are subsequently properly recognized as datetime.

Fixes https://github.com/dogsheep/healthkit-to-sqlite/issues/17

 healthkit-to-sqlite 197882382 pull     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/18/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
830901133 MDExOlB1bGxSZXF1ZXN0NTkyMzY0MjU1 16 Add a fallback ID, print if no ID found n8henrie 1234956 open 0     0 2021-03-13T13:38:29Z 2021-03-13T14:44:04Z   FIRST_TIME_CONTRIBUTOR dogsheep/healthkit-to-sqlite/pulls/16 healthkit-to-sqlite 197882382 pull     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/16/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
793907673 MDExOlB1bGxSZXF1ZXN0NTYxNTEyNTAz 15 added try / except to write_records ryancheley 9857779 open 0     0 2021-01-26T03:56:21Z 2021-01-26T03:56:21Z   FIRST_TIME_CONTRIBUTOR dogsheep/healthkit-to-sqlite/pulls/15

to keep the data write from failing if it came across an error during processing. In particular when trying to convert my HealthKit zip file (and that of my wife's) it would consistently error out with the following:

``` db.py 1709 insert_chunk result = self.db.execute(query, params)

db.py 226 execute return self.conn.execute(sql, parameters)

sqlite3.OperationalError: too many SQL variables

db.py 1709 insert_chunk result = self.db.execute(query, params)

db.py 226 execute return self.conn.execute(sql, parameters)

sqlite3.OperationalError: too many SQL variables

db.py 1709 insert_chunk result = self.db.execute(query, params)

db.py 226 execute return self.conn.execute(sql, parameters)

sqlite3.OperationalError: table rBodyMass has no column named metadata_HKWasUserEntered

healthkit-to-sqlite 8 <module> sys.exit(cli())

core.py 829 call return self.main(args, *kwargs)

core.py 782 main rv = self.invoke(ctx)

core.py 1066 invoke return ctx.invoke(self.callback, **ctx.params)

core.py 610 invoke return callback(args, *kwargs)

cli.py 57 cli convert_xml_to_sqlite(fp, db, progress_callback=bar.update, zipfile=zf)

utils.py 42 convert_xml_to_sqlite write_records(records, db)

utils.py 143 write_records db[table].insert_all(

db.py 1899 insert_all self.insert_chunk(

db.py 1720 insert_chunk self.insert_chunk(

db.py 1720 insert_chunk self.insert_chunk(

db.py 1714 insert_chunk result = self.db.execute(query, params)

db.py 226 execute return self.conn.execute(sql, parameters)

sqlite3.OperationalError: table rBodyMass has no column named metadata_HKWasUserEntered ```

Adding the try / except in the write_records seems to fix that issue.

healthkit-to-sqlite 197882382 pull     
{
    "url": "https://api.github.com/repos/dogsheep/healthkit-to-sqlite/issues/15/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);