issues

8 rows where repo = 205429375 sorted by updated_at descending

View and edit SQL

Suggested facets: user, comments, author_association, created_at (date), updated_at (date), closed_at (date)

type

state

repo

  • swarm-to-sqlite · 8
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason
648245071 MDU6SXNzdWU2NDgyNDUwNzE= 8 Error thrown: table photos has no column named hasSticker harperreed 18504 open 0     0 2020-06-30T14:54:37Z 2020-06-30T14:54:37Z   NONE  

While running swarm-to-sqlite it throws an error:

harper@:~/dogsheep/swarm$ swarm-to-sqlite checkins.db --save=checkins.json
Please provide your Foursquare OAuth token:
Importing 8127 checkins  [#################-------------------]   49%  00:01:52
Traceback (most recent call last):
File "/home/harper/.local/bin/swarm-to-sqlite", line 11, in <module>
    sys.exit(cli())
File "/home/harper/.local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
File "/home/harper/.local/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
File "/home/harper/.local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
File "/home/harper/.local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
File "/home/harper/.local/lib/python3.6/site-packages/swarm_to_sqlite/cli.py", line 73, in cli
    save_checkin(checkin, db)
File "/home/harper/.local/lib/python3.6/site-packages/swarm_to_sqlite/utils.py", line 94, in save_checkin
    photos_table.insert(photo, replace=True)
File "/home/harper/.local/lib/python3.6/site-packages/sqlite_utils/db.py", line 963, in insert
    alter = self.value_or_default("alter", alter)
File "/home/harper/.local/lib/python3.6/site-packages/sqlite_utils/db.py", line 1142, in insert_all
    def upsert_all(
sqlite3.OperationalError: table photos has no column named hasSticker

Where should i dig in?

swarm-to-sqlite 205429375 issue  
543355051 MDExOlB1bGxSZXF1ZXN0MzU3NjQwMTg2 6 don't break if source is missing mfa 78035 closed 0     1 2019-12-29T10:46:47Z 2020-03-28T02:28:11Z 2020-03-28T02:28:11Z CONTRIBUTOR dogsheep/swarm-to-sqlite/pulls/6

broke for me. very old checkins in 2010 had no source set.

swarm-to-sqlite 205429375 pull  
589491711 MDU6SXNzdWU1ODk0OTE3MTE= 7 Upgrade to sqlite-utils 2.x simonw 9599 closed 0     0 2020-03-28T02:24:51Z 2020-03-28T02:25:03Z 2020-03-28T02:25:03Z MEMBER   swarm-to-sqlite 205429375 issue  
487600595 MDU6SXNzdWU0ODc2MDA1OTU= 3 Option to fetch only checkins more recent than the current max checkin simonw 9599 closed 0     4 2019-08-30T17:46:45Z 2019-10-16T20:41:23Z 2019-10-16T20:39:59Z MEMBER  

The Foursquare checkins API supports "return every checkin occurring after this point" - I can pass it the maximum createdAt date currently stored in the database. This will allow for quick incremental fetches via a cron.

swarm-to-sqlite 205429375 issue  
487601121 MDU6SXNzdWU0ODc2MDExMjE= 4 Online tool for getting a Foursquare OAuth token simonw 9599 closed 0     1 2019-08-30T17:48:14Z 2019-08-31T18:07:26Z 2019-08-31T18:07:26Z MEMBER  

I will link to this from the documentation. See also this conversation on Twitter: https://twitter.com/simonw/status/1166822603023011840

I've decided to go with "copy and paste in a token" rather than hooking up a local web server that can have tokens passed to it.

swarm-to-sqlite 205429375 issue  
487721884 MDU6SXNzdWU0ODc3MjE4ODQ= 5 Treat Foursquare timestamps as UTC simonw 9599 closed 0     0 2019-08-31T02:44:47Z 2019-08-31T02:50:41Z 2019-08-31T02:50:41Z MEMBER  

Current test failure is due to timezone differences between my laptop and Circle CI:

https://circleci.com/gh/dogsheep/swarm-to-sqlite/3

E         Full diff:
E         - [{'created': '2018-07-01T04:48:19',
E         ?                           ^
E         + [{'created': '2018-07-01T02:48:19',
E         ?                           ^
E         'createdAt': 1530413299,

The timestamps I store in created should always be UTC.

swarm-to-sqlite 205429375 issue  
487598468 MDU6SXNzdWU0ODc1OTg0Njg= 2 --save option to dump checkins to a JSON file on disk simonw 9599 closed 0     1 2019-08-30T17:41:06Z 2019-08-31T02:40:21Z 2019-08-31T02:40:21Z MEMBER  

This is a complement to the --load option - mainly useful for development purposes.

(I'll rename --file to --load as part of this issue).

swarm-to-sqlite 205429375 issue  
487598042 MDU6SXNzdWU0ODc1OTgwNDI= 1 Implement code to pull checkins from the Foursquare API simonw 9599 closed 0     0 2019-08-30T17:40:02Z 2019-08-30T18:23:24Z 2019-08-30T18:23:24Z MEMBER  

The tool currently only works with a pre-prepared JSON file of checkins.

When called without options, it should prompt the user to paste in a Foursquare OAuth token.

The --token= option should work too, and should be backed up by an optional environment variable.

swarm-to-sqlite 205429375 issue  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);
Powered by Datasette · Query took 64.675ms · About: github-to-sqlite