issue_comments

Table actions

5 rows where issue = 1095570074 and "updated_at" is on date 2022-01-09 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 1

issue 1

  • `--batch-size 1` doesn't seem to commit for every item · 5 ✖

author_association 1

id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
1008234293 https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008234293 https://api.github.com/repos/simonw/sqlite-utils/issues/364 IC_kwDOCGYnMM48GG81 simonw 9599 2022-01-09T05:37:02Z 2022-01-09T05:37:02Z OWNER

Calling p.stdin.close() and then p.wait() terminates the subprocess.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `--batch-size 1` doesn't seem to commit for every item 1095570074  
1008233910 https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008233910 https://api.github.com/repos/simonw/sqlite-utils/issues/364 IC_kwDOCGYnMM48GG22 simonw 9599 2022-01-09T05:32:53Z 2022-01-09T05:35:45Z OWNER

This is strange. The following: ```pycon

import subprocess p = subprocess.Popen(["sqlite-utils", "insert", "/tmp/stream.db", "stream", "-", "--nl"], stdin=subprocess.PIPE) p.stdin.write(b'\n'.join(b'{"id": %s}' % str(i).encode("utf-8") for i in range(1000))) 11889

At this point /tmp/stream.db is still 0 bytes - but if I then run this:

p.stdin.close()

/tmp/stream.db is now 20K and contains the written data

`` No wait, mystery solved - I can addp.stdin.flush()instead ofp.stdin.close()` and the file suddenly jumps up in size.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `--batch-size 1` doesn't seem to commit for every item 1095570074  
1008216201 https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008216201 https://api.github.com/repos/simonw/sqlite-utils/issues/364 IC_kwDOCGYnMM48GCiJ simonw 9599 2022-01-09T02:34:12Z 2022-01-09T02:34:12Z OWNER

I can now write tests that look like this: https://github.com/simonw/sqlite-utils/blob/539f5ccd90371fa87f946018f8b77d55929e06db/tests/test_cli.py#L2024-L2030

Which means I can write a test that exercises this bug.

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `--batch-size 1` doesn't seem to commit for every item 1095570074  
1008214998 https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008214998 https://api.github.com/repos/simonw/sqlite-utils/issues/364 IC_kwDOCGYnMM48GCPW simonw 9599 2022-01-09T02:23:20Z 2022-01-09T02:23:20Z OWNER

Possible way of running the test: add this to sqlite_utils/cli.py:

python if __name__ == "__main__": cli() Now the tool can be run using python -m sqlite_utils.cli --help

Then in the test use subprocess to call sys.executable (the path to the current Python interpreter) and pass it -m sqlite_utils.cli to run the script!

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `--batch-size 1` doesn't seem to commit for every item 1095570074  
1008214406 https://github.com/simonw/sqlite-utils/issues/364#issuecomment-1008214406 https://api.github.com/repos/simonw/sqlite-utils/issues/364 IC_kwDOCGYnMM48GCGG simonw 9599 2022-01-09T02:18:21Z 2022-01-09T02:18:21Z OWNER

I'm having trouble figuring out the best way to write a unit test for this. Filed a relevant feature request for Click here: - https://github.com/pallets/click/issues/2171

 
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 `--batch-size 1` doesn't seem to commit for every item 1095570074  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);