issue_comments

Table actions

4 rows where author_association = "NONE" and user = 9020979 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: issue_url, created_at (date), updated_at (date)

user 1

  • hydrosquall · 4 ✖

author_association 1

  • NONE · 4 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1492777509 https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1492777509 https://api.github.com/repos/simonw/sqlite-utils/issues/235 IC_kwDOCGYnMM5Y-fol hydrosquall 9020979 2023-04-01T01:31:48Z 2023-04-01T01:31:48Z NONE

My current workaround is to use this library from a python script instead of as a CLI tool.

This lets me set the foreign key constraint at table creation time, instead of trying to modify an existing table. docs

I found this stackoverflow helpful, as it explained that Sqlite doesn't support modifying existing tables directly.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified 810618495  
1354192168 https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1354192168 https://api.github.com/repos/simonw/sqlite-utils/issues/235 IC_kwDOCGYnMM5Qt1Uo hydrosquall 9020979 2022-12-16T04:35:30Z 2022-12-16T04:35:38Z NONE

A related historical problem:

https://github.com/tekartik/sqflite/issues/525#issuecomment-714500720

I wonder if the version of Sqlite or Python for Intel chip have defensive mode disabled by default, whereas M1 chips versions have it enabled.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified 810618495  
1354160286 https://github.com/simonw/sqlite-utils/issues/235#issuecomment-1354160286 https://api.github.com/repos/simonw/sqlite-utils/issues/235 IC_kwDOCGYnMM5Qttie hydrosquall 9020979 2022-12-16T03:51:19Z 2022-12-16T03:52:13Z NONE

Hi @ryascott, thanks for sharing this! How did you upgrade your sqlite3 version? I'm running into this issue (also on an m1) with

Python ==3.10.7 sqlite3.sqlite_version==3.37.0 sqlite-utils==3.30

Unfortunately, 3.10.8 isn't listed in pyenv so I'm unable to install it.

For me, the trigger is trying to use the add-foreign-key command on its own:

bash sqlite-utils add-foreign-key library.db book_creators creator_id creators id

Some stackoverflow searching suggests that brew installing sqlite may fix it ( https://stackoverflow.com/questions/26345972/how-do-i-upgrade-the-sqlite-version-used-by-pythons-sqlite3-module-on-mac ), but I don't want to risk breaking the version of sqlite used by some other system, I'd only like to upgrade sqlite3 inside my current virtual environment.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 Extract columns cannot create foreign key relation: sqlite3.OperationalError: table sqlite_master may not be modified 810618495  
1208757153 https://github.com/dogsheep/github-to-sqlite/issues/51#issuecomment-1208757153 https://api.github.com/repos/dogsheep/github-to-sqlite/issues/51 IC_kwDODFdgUs5IDCuh hydrosquall 9020979 2022-08-09T00:29:44Z 2022-08-09T00:29:44Z NONE

I've been looking into how to to get this data out of Github (especially now there are "secondary rate limits" without an advertised allowance separate from the regular rate limits.

I've had decent success with the Airbyte github extractor (aside from one data quality issue https://github.com/airbytehq/airbyte/pull/15420 ). Airbyte splits data extraction between the GraphQL and REST endpoints depending on the resource type, but they're very comprehensive.

https://github.com/airbytehq/airbyte/blob/306a75ef5370728e0912cf52a1a898a530db0c90/airbyte-integrations/connectors/source-github/source_github/streams.py#L22-L122

Before this, I tried a few solutions in my own custom wrapper mentioned in this thread + its children https://github.com/PyGithub/PyGithub/issues/1989 , but they weren't working as expected.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 github-to-sqlite should handle rate limits better 703246031  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);