issue_comments

Table actions

5 rows where issue = 1373224657 sorted by updated_at descending

✖
✖

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 2

author_association 2

issue 1

  • `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float · 5 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1364141224 https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1364141224 https://api.github.com/repos/simonw/sqlite-utils/issues/488 IC_kwDOCGYnMM5RTySo ar-jan 1176293 2022-12-23T17:38:55Z 2022-12-23T17:38:55Z NONE

text columns containing empty strings should not be rewritten to null.

I would actually appreciate an option to do just that for text columns as well.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657  
1254033981 https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1254033981 https://api.github.com/repos/simonw/sqlite-utils/issues/488 IC_kwDOCGYnMM5Kvwo9 simonw 9599 2022-09-21T17:49:32Z 2022-09-21T17:50:10Z OWNER

It looks like SQLite has a SELECT NULLIF(value, '') function which returns null if that value is equal to ''.

We need to only apply that function to columns that we know to be of type integer or float though - text columns containing empty strings should not be rewritten to null.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657  
1254032378 https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1254032378 https://api.github.com/repos/simonw/sqlite-utils/issues/488 IC_kwDOCGYnMM5KvwP6 simonw 9599 2022-09-21T17:47:54Z 2022-09-21T17:47:54Z OWNER

New tests should go in: https://github.com/simonw/sqlite-utils/blob/main/tests/test_transform.py

I think the implementation fix needs to go near here: https://github.com/simonw/sqlite-utils/blob/0b315d3fa83c1584eaeec32f24912898621e437a/sqlite_utils/db.py#L1770-L1775

The trick is going to be teaching that generated SQL to know which columns are integer or float and to convert "" to null as part of that operation. 

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657  
1254029808 https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1254029808 https://api.github.com/repos/simonw/sqlite-utils/issues/488 IC_kwDOCGYnMM5Kvvnw simonw 9599 2022-09-21T17:45:20Z 2022-09-21T17:45:41Z OWNER

No, I'm going to say that this is a bug - it's WEIRD having a integer or float column containing an empty string. I'm OK changing that - I very much doubt anyone is relying on this functionality.

So no need for a new option here - just fixing the bug is sensible.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657  
1246971764 https://github.com/simonw/sqlite-utils/issues/488#issuecomment-1246971764 https://api.github.com/repos/simonw/sqlite-utils/issues/488 IC_kwDOCGYnMM5KU0d0 simonw 9599 2022-09-14T15:52:14Z 2022-09-14T15:52:14Z OWNER

Frustratingly I think this counts as a backwards-incompatible change.

Could have it be opt-in with a new option / method parameter, and then change it to the default if I release a sqlite-utils 4.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `sqlite-utils transform` should set empty strings to null when converting text columns to integer/float 1373224657  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);