{"id": 487598042, "node_id": "MDU6SXNzdWU0ODc1OTgwNDI=", "number": 1, "title": "Implement code to pull checkins from the Foursquare API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-08-30T17:40:02Z", "updated_at": "2019-08-30T18:23:24Z", "closed_at": "2019-08-30T18:23:24Z", "author_association": "MEMBER", "pull_request": null, "body": "The tool currently only works with a pre-prepared JSON file of checkins.\r\n\r\nWhen called without options, it should prompt the user to paste in a Foursquare OAuth token.\r\n\r\nThe `--token=` option should work too, and should be backed up by an optional environment variable.", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/1/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 487598468, "node_id": "MDU6SXNzdWU0ODc1OTg0Njg=", "number": 2, "title": "--save option to dump checkins to a JSON file on disk", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-08-30T17:41:06Z", "updated_at": "2019-08-31T02:40:21Z", "closed_at": "2019-08-31T02:40:21Z", "author_association": "MEMBER", "pull_request": null, "body": "This is a complement to the `--load` option - mainly useful for development purposes.\r\n\r\n(I'll rename `--file` to `--load` as part of this issue).", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/2/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 487600595, "node_id": "MDU6SXNzdWU0ODc2MDA1OTU=", "number": 3, "title": "Option to fetch only checkins more recent than the current max checkin", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2019-08-30T17:46:45Z", "updated_at": "2019-10-16T20:41:23Z", "closed_at": "2019-10-16T20:39:59Z", "author_association": "MEMBER", "pull_request": null, "body": "The Foursquare checkins API supports \"return every checkin occurring after this point\" - I can pass it the maximum createdAt date currently stored in the database. This will allow for quick incremental fetches via a cron.", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/3/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 487601121, "node_id": "MDU6SXNzdWU0ODc2MDExMjE=", "number": 4, "title": "Online tool for getting a Foursquare OAuth token", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-08-30T17:48:14Z", "updated_at": "2019-08-31T18:07:26Z", "closed_at": "2019-08-31T18:07:26Z", "author_association": "MEMBER", "pull_request": null, "body": "I will link to this from the documentation. See also this conversation on Twitter: https://twitter.com/simonw/status/1166822603023011840\r\n\r\nI've decided to go with \"copy and paste in a token\" rather than hooking up a local web server that can have tokens passed to it.", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/4/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 487721884, "node_id": "MDU6SXNzdWU0ODc3MjE4ODQ=", "number": 5, "title": "Treat Foursquare timestamps as UTC", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2019-08-31T02:44:47Z", "updated_at": "2019-08-31T02:50:41Z", "closed_at": "2019-08-31T02:50:41Z", "author_association": "MEMBER", "pull_request": null, "body": "Current test failure is due to timezone differences between my laptop and Circle CI:\r\n\r\nhttps://circleci.com/gh/dogsheep/swarm-to-sqlite/3\r\n\r\n```\r\nE Full diff:\r\nE - [{'created': '2018-07-01T04:48:19',\r\nE ? ^\r\nE + [{'created': '2018-07-01T02:48:19',\r\nE ? ^\r\nE 'createdAt': 1530413299,\r\n```\r\n\r\nThe timestamps I store in `created` should always be UTC.", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 543355051, "node_id": "MDExOlB1bGxSZXF1ZXN0MzU3NjQwMTg2", "number": 6, "title": "don't break if source is missing", "user": {"value": 78035, "label": "mfa"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2019-12-29T10:46:47Z", "updated_at": "2020-03-28T02:28:11Z", "closed_at": "2020-03-28T02:28:11Z", "author_association": "CONTRIBUTOR", "pull_request": "dogsheep/swarm-to-sqlite/pulls/6", "body": "broke for me. very old checkins in 2010 had no source set.", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/6/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 589491711, "node_id": "MDU6SXNzdWU1ODk0OTE3MTE=", "number": 7, "title": "Upgrade to sqlite-utils 2.x", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-03-28T02:24:51Z", "updated_at": "2020-03-28T02:25:03Z", "closed_at": "2020-03-28T02:25:03Z", "author_association": "MEMBER", "pull_request": null, "body": "", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/7/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 648245071, "node_id": "MDU6SXNzdWU2NDgyNDUwNzE=", "number": 8, "title": "Error thrown: table photos has no column named hasSticker", "user": {"value": 18504, "label": "harperreed"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-06-30T14:54:37Z", "updated_at": "2020-10-12T20:35:06Z", "closed_at": "2020-10-12T20:25:24Z", "author_association": "NONE", "pull_request": null, "body": "While running `swarm-to-sqlite` it throws an error:\r\n\r\n harper@:~/dogsheep/swarm$ swarm-to-sqlite checkins.db --save=checkins.json\r\n Please provide your Foursquare OAuth token:\r\n Importing 8127 checkins [#################-------------------] 49% 00:01:52\r\n Traceback (most recent call last):\r\n File \"/home/harper/.local/bin/swarm-to-sqlite\", line 11, in \r\n sys.exit(cli())\r\n File \"/home/harper/.local/lib/python3.6/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/home/harper/.local/lib/python3.6/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/home/harper/.local/lib/python3.6/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/home/harper/.local/lib/python3.6/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/home/harper/.local/lib/python3.6/site-packages/swarm_to_sqlite/cli.py\", line 73, in cli\r\n save_checkin(checkin, db)\r\n File \"/home/harper/.local/lib/python3.6/site-packages/swarm_to_sqlite/utils.py\", line 94, in save_checkin\r\n photos_table.insert(photo, replace=True)\r\n File \"/home/harper/.local/lib/python3.6/site-packages/sqlite_utils/db.py\", line 963, in insert\r\n alter = self.value_or_default(\"alter\", alter)\r\n File \"/home/harper/.local/lib/python3.6/site-packages/sqlite_utils/db.py\", line 1142, in insert_all\r\n def upsert_all(\r\n sqlite3.OperationalError: table photos has no column named hasSticker\r\n\r\nWhere should i dig in?", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/8/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 673602857, "node_id": "MDU6SXNzdWU2NzM2MDI4NTc=", "number": 9, "title": "Define a view that displays photos correctly", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2020-08-05T14:53:39Z", "updated_at": "2020-08-05T14:53:39Z", "closed_at": null, "author_association": "MEMBER", "pull_request": null, "body": "The `photos` table stores data like this:\r\n\r\nid | createdAt | source | prefix | suffix | width | height | visibility | created\u00a0\u25b2 | user\r\n-- | -- | -- | -- | -- | -- | -- | -- | -- | --\r\n5e12c9708506bc000840262a | January 06, 2020 - 05:45:20 UTC | Swarm for iOS\u00a01 | https://fastly.4sqi.net/img/general/ | /15889193_AXxGk4I1nbzUZuyYqObgbXdJNyEHiwj6AUDq0tPZWtw.jpg | 1920 | 1440 | public | 2020-01-06T05:45:20 | 15889193\r\n\r\nThe photo URL can be derived from those pieces - define a SQL view which does that (using `datasette-json-html` to display the pictures)", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/9/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 719637258, "node_id": "MDExOlB1bGxSZXF1ZXN0NTAxNzkxNjYz", "number": 10, "title": "Update utils.py to fix sqlite3.OperationalError", "user": {"value": 29426418, "label": "mattiaborsoi"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2020-10-12T20:17:53Z", "updated_at": "2020-10-12T20:25:10Z", "closed_at": "2020-10-12T20:25:09Z", "author_association": "CONTRIBUTOR", "pull_request": "dogsheep/swarm-to-sqlite/pulls/10", "body": "Fixes the errors:\r\n- sqlite3.OperationalError: table posts has no column named text\r\n- sqlite3.OperationalError: table photos has no column named hasSticker\r\n\r\nThat will cause sqlite-utils to notice if there's a missing column and add it. As recommended by @simonw", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/10/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 743400216, "node_id": "MDU6SXNzdWU3NDM0MDAyMTY=", "number": 11, "title": "Error thrown: sqlite3.OperationalError: table users has no column named lastName", "user": {"value": 61791, "label": "beaugunderson"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 2, "created_at": "2020-11-16T01:21:18Z", "updated_at": "2021-01-18T04:35:22Z", "closed_at": "2021-01-18T04:35:22Z", "author_association": "NONE", "pull_request": null, "body": "Just installed `swarm-to-sqlite-0.3.2` and tried according to the docs:\r\n\r\n```\r\nTraceback (most recent call last):\r\n File \"/usr/local/bin/swarm-to-sqlite\", line 8, in \r\n sys.exit(cli())\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/usr/local/lib/python3.9/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/usr/local/lib/python3.9/site-packages/swarm_to_sqlite/cli.py\", line 73, in cli\r\n save_checkin(checkin, db)\r\n File \"/usr/local/lib/python3.9/site-packages/swarm_to_sqlite/utils.py\", line 82, in save_checkin\r\n checkins_table.m2m(\"users\", user, m2m_table=\"likes\", pk=\"id\")\r\n File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1914, in m2m\r\n id = other_table.insert(record, pk=pk, replace=True).last_pk\r\n File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1647, in insert\r\n return self.insert_all(\r\n File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1765, in insert_all\r\n self.insert_chunk(\r\n File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 1575, in insert_chunk\r\n result = self.db.execute(query, params)\r\n File \"/usr/local/lib/python3.9/site-packages/sqlite_utils/db.py\", line 200, in execute\r\n return self.conn.execute(sql, parameters)\r\nsqlite3.OperationalError: table users has no column named lastName\r\n```", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"} {"id": 951817328, "node_id": "MDU6SXNzdWU5NTE4MTczMjg=", "number": 12, "title": "403 when getting token", "user": {"value": 285352, "label": "treyhunner"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 1, "created_at": "2021-07-23T18:43:26Z", "updated_at": "2021-10-12T18:31:57Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "I tried to use https://your-foursquare-oauth-token.glitch.me/ to get my Swarm auth token and got a 403 after I clicked the Allow button:\r\n\r\n![image](https://user-images.githubusercontent.com/285352/126826478-60e53614-263d-40bb-9f1d-c1a676644eb0.png)\r\n\r\nI'm not sure if this is the right repo to report this in", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/12/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1373210675, "node_id": "I_kwDODD6af85R2Ygz", "number": 13, "title": "fails before generating views. ERR: table sqlite_master may not be modified", "user": {"value": 116795, "label": "pax"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 4, "created_at": "2022-09-14T15:41:50Z", "updated_at": "2023-04-11T03:46:17Z", "closed_at": null, "author_association": "NONE", "pull_request": null, "body": "generates checkins.db but seems to fail before generating views \r\n\r\nnote: it worked on an Ubuntu WSL but fails on macOS 12.5.1\r\n\r\nlater edit: I suspect this is a problem with my local set-up, `dogsheep-beta index` also throws the same error\r\n\r\nfull error:\r\n\r\n Importing 2591 checkins [###################################-] 98% 00:00:00\r\n Traceback (most recent call last):\r\n File \"/Users/pax/devbox/envAll/bin/swarm-to-sqlite\", line 8, in \r\n sys.exit(cli())\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 829, in __call__\r\n return self.main(*args, **kwargs)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 782, in main\r\n rv = self.invoke(ctx)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 1066, in invoke\r\n return ctx.invoke(self.callback, **ctx.params)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/click/core.py\", line 610, in invoke\r\n return callback(*args, **kwargs)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/swarm_to_sqlite/cli.py\", line 77, in cli\r\n ensure_foreign_keys(db)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/swarm_to_sqlite/utils.py\", line 145, in ensure_foreign_keys\r\n db[fk.table].add_foreign_key(fk.column, fk.other_table, fk.other_column)\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/sqlite_utils/db.py\", line 2123, in add_foreign_key\r\n self.db.add_foreign_keys([(self.name, column, other_table, other_column)])\r\n File \"/Users/pax/devbox/envAll/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1086, in add_foreign_keys\r\n cursor.execute(\r\n sqlite3.OperationalError: table sqlite_master may not be modified", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/13/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null} {"id": 1393330070, "node_id": "PR_kwDODD6af84__DNJ", "number": 14, "title": "Photo links", "user": {"value": 6782721, "label": "redmanmale"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2022-10-01T09:44:15Z", "updated_at": "2022-11-18T17:10:49Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/swarm-to-sqlite/pulls/14", "body": "* add to `checkin_details` view new column for a calculated photo links\r\n* supported multiple links split by newline\r\n* create `events` table if there's no events in the history to avoid SQL errors\r\n\r\nFixes #9.", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/14/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null} {"id": 1661617056, "node_id": "I_kwDODD6af85jCkOg", "number": 15, "title": "ambiguous column name: createdAt - on checkin_details view", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 0, "created_at": "2023-04-11T01:07:47Z", "updated_at": "2023-04-11T03:16:37Z", "closed_at": "2023-04-11T03:16:37Z", "author_association": "MEMBER", "pull_request": null, "body": "It looks like Swarm changed their schema and now both `venues` and `checkins` have `createdAt` fields.\r\n\r\nWhich breaks this view: https://github.com/dogsheep/swarm-to-sqlite/blob/719b6e96a016d0ca8b316d3bed9c2a7a0cb499ee/swarm_to_sqlite/utils.py#L171-L188", "repo": {"value": 205429375, "label": "swarm-to-sqlite"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/15/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}