github
id | node_id | number | title | user | state | locked | assignee | milestone | comments | created_at | updated_at | closed_at | author_association | pull_request | body | repo | type | active_lock_reason | performed_via_github_app | reactions | draft | state_reason |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1617962395 | I_kwDOJHON9s5gcCWb | 10 | Include schema in README | 9599 | closed | 0 | 0 | 2023-03-09T20:38:59Z | 2023-03-09T20:48:18Z | 2023-03-09T20:48:18Z | MEMBER | As seen in other tools like https://github.com/simonw/git-history | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1617823309 | I_kwDOJHON9s5gbgZN | 8 | Increase performance using macnotesapp | 41546558 | closed | 0 | 1 | 2023-03-09T18:51:05Z | 2023-03-14T22:00:22Z | 2023-03-14T22:00:21Z | NONE | Neat project! You can probably increase performance using my python interface to Notes, [macnotesapp](https://github.com/RhetTbull/macnotesapp), which uses Scripting Bridge and bulk queries for much better performance than AppleScript. Another related project is [PyXA](https://github.com/SKaplanOfficial/PyXA) which uses Scripting Bridge to access Notes (and many other apps) and can return all the notes at once as opposed to calling AppleScript for each note. macnotesapp allows you to access multiple accounts and folders as well. ```python from macnotesapp import NotesApp # NotesApp() provides interface to Notes.app notesapp = NotesApp() # Get list of notes (Note objects for each note) notes = notesapp.notes() note = notes[0] print( note.id, note.account, note.folder, note.name, note.body, note.plaintext, note.password_protected, ) print(note.asdict()) ``` | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1617769847 | I_kwDOJHON9s5gbTV3 | 7 | Folder support | 9599 | closed | 0 | 6 | 2023-03-09T18:21:33Z | 2023-03-09T20:48:18Z | 2023-03-09T20:48:18Z | MEMBER | Notes can live in folders. These relationships should be exported too. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1616422013 | I_kwDOJHON9s5gWKR9 | 3 | `apple-notes-to-sqlite --dump` option | 9599 | closed | 0 | 0 | 2023-03-09T05:05:49Z | 2023-03-09T05:06:14Z | 2023-03-09T05:06:14Z | MEMBER | Option that doesn't write to the database at all, it just outputs all the notes to stdout as newline-delimited JSON. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1616354999 | I_kwDOJHON9s5gV563 | 2 | First working version | 9599 | closed | 0 | 7 | 2023-03-09T03:53:00Z | 2023-03-09T05:10:22Z | 2023-03-09T05:10:22Z | MEMBER | It's going to shell out to `osascript` as seen in: - #1 I'm going with that option because https://appscript.sourceforge.io/status.html warns against the other potential methods: > Apple eliminated its Mac Automation department in 2016. The future of AppleScript and its related technologies is unclear. Caveat emptor. But `osascript` looks pretty stable to me. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1616347574 | I_kwDOJHON9s5gV4G2 | 1 | Initial proof of concept with ChatGPT | 9599 | closed | 0 | 3 | 2023-03-09T03:44:39Z | 2023-03-09T03:51:55Z | 2023-03-09T03:51:55Z | MEMBER | I'm using ChatGPT to figure out enough AppleScript to get at my notes data. | 611552758 | issue | { "url": "https://api.github.com/repos/dogsheep/apple-notes-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
978743426 | MDU6SXNzdWU5Nzg3NDM0MjY= | 13 | xml.etree.ElementTree.ParseError: not well-formed (invalid token) | 9599 | closed | 0 | 4 | 2021-08-25T05:48:21Z | 2021-08-26T18:45:13Z | 2021-08-26T18:45:13Z | MEMBER | Got this error today: ``` (evernote-to-sqlite) /tmp % evernote-to-sqlite enex evernote.db simonwillison\'s\ notebook.enex Importing from ENEX [######------------------------------] 17% Traceback (most recent call last): File "/Users/simon/.local/bin/evernote-to-sqlite", line 8, in <module> sys.exit(cli()) File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1137, in __call__ return self.main(*args, **kwargs) File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1062, in main rv = self.invoke(ctx) File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1668, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/click/core.py", line 763, in invoke return __callback(*args, **kwargs) File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/cli.py", line 31, in enex save_note(db, note) File "/Users/simon/.local/pipx/venvs/evernote-to-sqlite/lib/python3.9/site-packages/evernote_to_sqlite/utils.py", line 36, in save_note content = ET.tostring(ET.fromstring(content_xml)).decode("utf-8") File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/xml/etree/ElementTree.py", line 1347, in XML parser.feed(text) xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 2, column 132 ``` | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
792851444 | MDU6SXNzdWU3OTI4NTE0NDQ= | 11 | XML parse error | 3613583 | closed | 0 | 2 | 2021-01-24T17:38:54Z | 2021-02-11T21:18:58Z | 2021-02-11T21:18:48Z | NONE | I am on Windows 10 using Windows Subsystem for Linux, Python 3.8. I installed evernote-to-sqlite via pipx (in a venv). I tried using enex files from the latest version of Evernote for Windows (10.6.9 which only lets you export 50 notes at a time) and from Legacy Evernote (6.25.2.9198 which lets you export all your notes at once). The enex file from latest evernote gives this error: File "/usr/lib/python3.8/xml/etree/ElementTree.py", line 1320, in XML parser.feed(text) xml.etree.ElementTree.ParseError: XML or text declaration not at start of entity: line 2, column 6 The enex file from Legacy Evernote gives this error: File "/home/david/.local/pipx/venvs/evernote-to-sqlite/lib/python3.8/site-packages/evernote_to_sqlite/utils.py", line 28, in save_note updated = note.find("updated").text AttributeError: 'NoneType' object has no attribute 'text' | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
770712149 | MDExOlB1bGxSZXF1ZXN0NTQyNDA2OTEw | 10 | BugFix for encoding and not update info. | 1277270 | closed | 0 | 1 | 2020-12-18T08:58:54Z | 2021-02-11T22:37:56Z | 2021-02-11T22:37:56Z | NONE | dogsheep/evernote-to-sqlite/pulls/10 | Bugfix 1: Traceback (most recent call last): File "d:\anaconda3\lib\runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "d:\anaconda3\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "D:\Anaconda3\Scripts\evernote-to-sqlite.exe\__main__.py", line 7, in <module> File "d:\anaconda3\lib\site-packages\click\core.py", line 829, in __call__ File "d:\anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "d:\anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) return ctx.invoke(self.callback, **ctx.params) File "d:\anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs) File "d:\anaconda3\lib\site-packages\evernote_to_sqlite\cli.py", line 30, in enex for tag, note in find_all_tags(fp, ["note"], progress_callback=bar.update): File "d:\anaconda3\lib\site-packages\evernote_to_sqlite\utils.py", line 11, in find_all_tags chunk = fp.read(1024 * 1024) UnicodeDecodeError: 'gbk' codec can't decode byte 0xa4 in position 383: illegal multibyte sequence Bugfix 2: Traceback (most recent call last): File "D:\Anaconda3\Scripts\evernote-to-sqlite-script.py", line 33, in <module> sys.exit(load_entry_point('evernote-to-sqlite==0.3', 'console_scripts', 'evernote-to-sqlite')()) File "D:\Anaconda3\lib\site-packages\click\core.py", line 829, in __call__ return self.main(*args, **kwargs) File "D:\Anaconda3\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx) File "D:\Anaconda3\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "D:\Anaconda3\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "D:\Anaconda3\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kw… | 303218369 | pull | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
748372469 | MDU6SXNzdWU3NDgzNzI0Njk= | 9 | ParseError: undefined entity š | 4028322 | closed | 0 | 1 | 2020-11-22T23:04:35Z | 2021-02-11T22:10:55Z | 2021-02-11T22:10:55Z | CONTRIBUTOR | I encountered a parse error if the enex file contained š or Run command: evernote-to-sqlite enex evernote.db evernote.enex ``` Traceback (most recent call last): ... File "evernote_to_sqlite/cli.py", line 31, in enex save_note(db, note) File "evernote_to_sqlite/utils.py", line 35, in save_note content = ET.tostring(ET.fromstring(content_xml)).decode("utf-8") File "/usr/lib/python3.8/xml/etree/ElementTree.py", line 1320, in XML parser.feed(text) xml.etree.ElementTree.ParseError: undefined entity š: line 3, column 35 ``` Workaround: ``` sed -i 's/š//g' evernote.enex sed -i 's/ //g' evernote.enex ``` | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
748370021 | MDExOlB1bGxSZXF1ZXN0NTI1MzcxMDI5 | 8 | fix import error if note has no "updated" element | 4028322 | closed | 0 | 0 | 2020-11-22T22:51:05Z | 2021-02-11T22:34:06Z | 2021-02-11T22:34:06Z | CONTRIBUTOR | dogsheep/evernote-to-sqlite/pulls/8 | I got the following error when executing evernote-to-sqlite enex evernote.db evernote.enex ``` ... File "evernote_to_sqlite/cli.py", line 31, in enex save_note(db, note) File "evernote_to_sqlite/utils.py", line 28, in save_note updated = note.find("updated").text AttributeError: 'NoneType' object has no attribute 'text' ``` Seems that in some cases the updated element is not added to the note, this is a part of the problematic note: ``` <created>20201019T074518Z</created> <note-attributes> <source>web.clip7</source> <source-application>webclipper.evernote</source-application> </note-attributes> ``` | 303218369 | pull | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
743297582 | MDU6SXNzdWU3NDMyOTc1ODI= | 7 | evernote-to-sqlite on windows 10 give this error: TypeError: insert() got an unexpected keyword argument 'replace' | 42387931 | closed | 0 | 1 | 2020-11-15T16:57:28Z | 2021-02-11T22:13:17Z | 2021-02-11T22:13:17Z | NONE | running evernote-to-sqlite 0.2 on windows 10. Command: evernote-to-sqlite enex evernote.db MyNotes.enex I get the followinng error: File "C:\Users\marti\AppData\Roaming\Python\Python38\site-packages\evernote_to_sqlite\utils.py", line 46, in save_note note_id = db["notes"].insert(row, hash_id="id", replace=True, alter=True).last_pk TypeError: insert() got an unexpected keyword argument 'replace' Removing replace=True, Leads to below error: note_id = db["notes"].insert(row, hash_id="id", alter=True).last_pk File "C:\Users\marti\AppData\Roaming\Python\Python38\site-packages\sqlite_utils\db.py", line 924, in insert return self.insert_all( File "C:\Users\marti\AppData\Roaming\Python\Python38\site-packages\sqlite_utils\db.py", line 1046, in insert_all result = self.db.conn.execute(sql, values) sqlite3.IntegrityError: UNIQUE constraint failed: notes.id | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
718949182 | MDU6SXNzdWU3MTg5NDkxODI= | 6 | Better handling of OCR data | 9599 | closed | 0 | 2 | 2020-10-11T23:20:52Z | 2020-10-12T00:04:10Z | 2020-10-12T00:04:10Z | MEMBER | > I haven't done the FTS on OCR yet. I'm going to move that to another ticket because it requires more thought. _Originally posted by @simonw in https://github.com/dogsheep/evernote-to-sqlite/issues/4#issuecomment-706784028_ | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
718938508 | MDU6SXNzdWU3MTg5Mzg1MDg= | 4 | Configure FTS + add an index on the date columns | 9599 | closed | 0 | 2 | 2020-10-11T22:14:40Z | 2020-10-11T23:41:29Z | 2020-10-11T23:41:29Z | MEMBER | Sort by date descending is likely the most common way of sorting, so that column should be indexed. Also add FTS configuration for both notes and the OCR column on resources. | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
718938321 | MDU6SXNzdWU3MTg5MzgzMjE= | 3 | Use a content hash for the note IDs | 9599 | closed | 0 | 0 | 2020-10-11T22:13:46Z | 2020-10-11T23:15:04Z | 2020-10-11T23:15:04Z | MEMBER | Without a GUID note IDs are pretty ineffective, but using a hash of the contents will at least avoid creating identical duplicates in the future. https://sqlite-utils.readthedocs.io/en/stable/python-api.html#setting-an-id-based-on-the-hash-of-the-row-contents | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
718938046 | MDU6SXNzdWU3MTg5MzgwNDY= | 2 | Convert dates to a better format | 9599 | closed | 0 | 0 | 2020-10-11T22:12:33Z | 2020-10-11T23:15:03Z | 2020-10-11T23:15:03Z | MEMBER | They currently look like this: https://github.com/dogsheep/evernote-to-sqlite/blob/9d8efd17580f6ddf76745c145d1e69dd24e52b64/tests/test_evernote_to_sqlite.py#L35-L36 | 303218369 | issue | { "url": "https://api.github.com/repos/dogsheep/evernote-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1871935751 | I_kwDOD079W85vk3kH | 40 | ImportError: cannot import name 'formatargspec' from 'inspect' | 36752421 | closed | 0 | 0 | 2023-08-29T15:36:31Z | 2023-08-31T03:18:07Z | 2023-08-31T03:18:06Z | NONE | I get the following error when running "pip3 install dogsheep-photos" " from inspect import ismethod, isclass, formatargspec ImportError: cannot import name 'formatargspec' from 'inspect' (/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/inspect.py). Did you mean: 'formatargvalues'?" Python 3.12.0rc1 sqlite 3.43.0 datasette, version 0.64.3 | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/40/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1827427757 | PR_kwDOD079W85WtUKG | 38 | photos-to-sql not found? | 319473 | closed | 0 | 2 | 2023-07-29T09:59:42Z | 2023-07-29T10:01:27Z | 2023-07-29T10:01:23Z | FIRST_TIME_CONTRIBUTOR | dogsheep/dogsheep-photos/pulls/38 | I wonder if `photos-to-sql` is an old name for `dogsheep-photos`, because I can't find it anywhere. I can't actually get this command to work (`sqlite3.OperationalError: no such table: attached.ZGENERICASSET` thrown) but I don't think that's related | 256834907 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/38/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
638375985 | MDExOlB1bGxSZXF1ZXN0NDM0MTYyMzE2 | 29 | Fixed bug in SQL query for photo scores | 41546558 | closed | 0 | 1 | 2020-06-14T15:39:22Z | 2020-12-04T22:32:36Z | 2020-12-04T22:32:27Z | CONTRIBUTOR | dogsheep/dogsheep-photos/pulls/29 | The join on ZCOMPUTEDASSETATTRIBUTES used the wrong columns. In most of the Photos database tables, table.ZASSET joins with ZGENERICASSET.Z_PK | 256834907 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/29/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
621444763 | MDU6SXNzdWU2MjE0NDQ3NjM= | 26 | Rename project to dogsheep-photos | 9599 | closed | 0 | 8 | 2020-05-20T04:12:34Z | 2020-05-20T04:31:02Z | 2020-05-20T04:30:40Z | MEMBER | `photos-to-sqlite` doesn't really capture the full scope of this project anymore. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/26/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
621332242 | MDU6SXNzdWU2MjEzMzIyNDI= | 25 | Create a public demo | 9599 | closed | 0 | 5 | 2020-05-19T22:47:20Z | 2020-05-21T22:26:16Z | 2020-05-20T05:54:18Z | MEMBER | So I can show people what this does, using some of my photos. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/25/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
621280529 | MDU6SXNzdWU2MjEyODA1Mjk= | 23 | create-subset command for creating a publishable subset of a photos database | 9599 | closed | 0 | 1 | 2020-05-19T20:58:20Z | 2020-05-19T22:32:48Z | 2020-05-19T22:32:37Z | MEMBER | I want to share a subset of my photos, without sharing everything. Idea: $ photos-to-sqlite create-subset photos.db public.db "select sha256 from ... where ..." So the command takes a SQL query that returns sha256 hashes, then creates a new file called `public.db` containing just the data corresponding to those photos. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/23/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
615474990 | MDU6SXNzdWU2MTU0NzQ5OTA= | 21 | bpylist.archiver.CircularReference: archive has a cycle with uid(13) | 9599 | closed | 0 | 11 | 2020-05-10T20:58:06Z | 2020-12-19T07:44:49Z | 2020-05-10T21:57:13Z | MEMBER | ``` % python -i $(which photos-to-sqlite) apple-photos photos.db Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py", line 611, in place return self._place # pylint: disable=access-member-before-definition AttributeError: 'PhotoInfo' object has no attribute '_place' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/bin/photos-to-sqlite", line 11, in <module> load_entry_point('photos-to-sqlite', 'console_scripts', 'photos-to-sqlite')() File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/Users/simon/Dropbox/Development/photos-to-sqlite/photos_to_sqlite/cli.py", line 249, in apple_photos photo_row = osxphoto_to_row(sha256, photo) File "/Users/simon/Dropbox/Development/photos-to-sqlite/photos_to_sqlite/utils.py", line 91, in osxphoto_to_row place = photo.place File "/Users/simon/.local/share/virtualenvs/photos-to-sqlite-0uGSHd6e/lib/python3.8/site-packages/osxphotos/photoinfo.py", line … | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/21/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613006393 | MDU6SXNzdWU2MTMwMDYzOTM= | 20 | Ability to serve thumbnailed Apple Photo from its place on disk | 9599 | closed | 0 | 10 | 2020-05-06T02:17:50Z | 2020-05-25T20:14:22Z | 2020-05-25T20:09:41Z | MEMBER | A custom Datasette plugin that can be run locally on a Mac laptop which knows how to serve photos such that they can be seen in the browser. _Originally posted by @simonw in https://github.com/dogsheep/photos-to-sqlite/issues/19#issuecomment-624406285_ | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613002220 | MDU6SXNzdWU2MTMwMDIyMjA= | 19 | apple-photos command should work even if upload has not run | 9599 | closed | 0 | 1 | 2020-05-06T02:02:25Z | 2020-05-19T20:59:59Z | 2020-05-19T20:59:59Z | MEMBER | I want people to be able to query their Apple Photos metadata without having to first run `upload` to upload all of their files to their own S3 bucket. To do this I can have `apple-photos` calculate SHA256 hashes of each photo if the `uploads` table does not yet exist (or does not contain that photo). | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612860531 | MDU6SXNzdWU2MTI4NjA1MzE= | 17 | Only install osxphotos if running on macOS | 9599 | closed | 0 | 3 | 2020-05-05T20:03:26Z | 2020-05-05T20:20:05Z | 2020-05-05T20:11:23Z | MEMBER | The build is broken right now because you can't `pip install osxphotos` on Ubuntu. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
612151767 | MDU6SXNzdWU2MTIxNTE3Njc= | 15 | Expose scores from ZCOMPUTEDASSETATTRIBUTES | 9599 | closed | 0 | 7 | 2020-05-04T20:36:07Z | 2020-12-20T04:44:22Z | 2020-05-05T00:11:45Z | MEMBER | The Apple Photos database has a `ZCOMPUTEDASSETATTRIBUTES` that looks absurdly interesting... it has calculated scores for every photo: <img width="273" alt="Photos__ZCOMPUTEDASSETATTRIBUTES" src="https://user-images.githubusercontent.com/9599/81011044-1f0e3d00-8e0c-11ea-84b6-f302b09e7bc9.png"> | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
606032950 | MDU6SXNzdWU2MDYwMzI5NTA= | 11 | Try running S3 uploads in a thread pool | 9599 | closed | 0 | 0 | 2020-04-24T04:34:31Z | 2020-04-24T16:45:41Z | 2020-04-24T16:45:41Z | MEMBER | Since #10 provided such a speedup, can the same thing be done for the actual uploads? http://ls.pwd.io/2013/06/parallel-s3-uploads-using-boto-and-threads-in-python/ suggests it can really help performance. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
606028272 | MDU6SXNzdWU2MDYwMjgyNzI= | 10 | Speed up hashing step using threads | 9599 | closed | 0 | 0 | 2020-04-24T04:20:08Z | 2020-04-24T04:32:35Z | 2020-04-24T04:32:35Z | MEMBER | This TODO from the code: https://github.com/dogsheep/photos-to-sqlite/blob/2e7f2c67cc18b02c75bb64992a05b0196e507252/photos_to_sqlite/cli.py#L82-L90 | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
605938063 | MDU6SXNzdWU2MDU5MzgwNjM= | 9 | upload command should be resumable, should only upload photos not already uploaded | 9599 | closed | 0 | 2 | 2020-04-23T23:31:08Z | 2020-04-23T23:39:14Z | 2020-04-23T23:39:14Z | MEMBER | Follow on from #4. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
605147638 | MDU6SXNzdWU2MDUxNDc2Mzg= | 8 | Should I have used MD5 instead of SHA256? | 9599 | closed | 0 | 2 | 2020-04-23T00:02:08Z | 2020-04-23T00:03:35Z | 2020-04-23T00:03:35Z | MEMBER | https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html > Objects created by the PUT Object, POST Object, or Copy operation, or through the AWS Management Console, and are encrypted by SSE-S3 or plaintext, have ETags that are an MD5 digest of their object data. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
602575575 | MDU6SXNzdWU2MDI1NzU1NzU= | 6 | Add progress bar to upload command | 9599 | closed | 0 | 2 | 2020-04-18T23:32:41Z | 2020-04-19T00:15:24Z | 2020-04-19T00:15:24Z | MEMBER | Upload was added in #4 | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
602551638 | MDU6SXNzdWU2MDI1NTE2Mzg= | 5 | photos-to-sqlite s3-auth command | 9599 | closed | 0 | 1 | 2020-04-18T21:05:25Z | 2020-04-18T21:08:44Z | 2020-04-18T21:08:44Z | MEMBER | Modeled on `github-to-sqlite auth` - prompts the user for their S3 credentials and saves them to `auth.json`. | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
602533539 | MDU6SXNzdWU2MDI1MzM1Mzk= | 4 | Upload all my photos to a secure S3 bucket | 9599 | closed | 0 | 5324096 | 14 | 2020-04-18T19:24:50Z | 2020-04-18T21:58:11Z | 2020-04-18T21:57:13Z | MEMBER | - [x] Create a bucket with bucket credentials - [x] Programmatically upload some recent photos to it (from a notebook) - [x] Turn this into a script | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
602533352 | MDU6SXNzdWU2MDI1MzMzNTI= | 2 | Ability to convert HEIC images to JPEG | 9599 | closed | 0 | 5324096 | 1 | 2020-04-18T19:23:43Z | 2020-04-28T16:47:21Z | 2020-04-28T16:47:21Z | MEMBER | 256834907 | issue | { "url": "https://api.github.com/repos/dogsheep/dogsheep-photos/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
585526292 | MDU6SXNzdWU1ODU1MjYyOTI= | 1 | Set up full text search | 9599 | closed | 0 | 1 | 2020-03-21T15:57:35Z | 2020-03-21T19:47:46Z | 2020-03-21T19:45:52Z | MEMBER | Should run against `title` and `text` in `items`, and `about` and `id` in `users`. | 248903544 | issue | { "url": "https://api.github.com/repos/dogsheep/hacker-news-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
842765105 | MDExOlB1bGxSZXF1ZXN0NjAyMjYxMDky | 6 | Add testres-db tool | 1151557 | closed | 0 | 1 | 2021-03-28T15:43:23Z | 2022-02-16T05:12:05Z | 2022-02-16T05:12:05Z | NONE | dogsheep/dogsheep.github.io/pulls/6 | 214746582 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
558715564 | MDExOlB1bGxSZXF1ZXN0MzcwMDI0Njk3 | 4 | Add beeminder-to-sqlite | 706257 | closed | 0 | 0 | 2020-02-02T15:51:36Z | 2020-10-12T00:36:16Z | 2020-10-12T00:36:16Z | CONTRIBUTOR | dogsheep/dogsheep.github.io/pulls/4 | 214746582 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
543717994 | MDExOlB1bGxSZXF1ZXN0MzU3OTc0MzI2 | 3 | Add todoist-to-sqlite | 706257 | closed | 0 | 0 | 2019-12-30T04:02:59Z | 2020-10-12T00:35:58Z | 2020-10-12T00:35:57Z | CONTRIBUTOR | dogsheep/dogsheep.github.io/pulls/3 | Really enjoying getting into the dogsheep/datasette ecosystem. I made a downloader for Todoist, and I think/hope others might find this useful | 214746582 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
519979091 | MDExOlB1bGxSZXF1ZXN0MzM4NjQ3Mzc4 | 1 | Add parkrun-to-sqlite | 1101318 | closed | 0 | 0 | 2019-11-08T12:05:32Z | 2020-10-12T00:35:16Z | 2020-10-12T00:35:16Z | CONTRIBUTOR | dogsheep/dogsheep.github.io/pulls/1 | 214746582 | pull | { "url": "https://api.github.com/repos/dogsheep/dogsheep.github.io/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
1795187493 | I_kwDODLZ_YM5rAGMl | 12 | Switch to pyproject.toml | 9599 | closed | 0 | 2 | 2023-07-09T01:06:56Z | 2023-07-09T01:19:43Z | 2023-07-09T01:19:42Z | MEMBER | First of my CLI tools to use https://til.simonwillison.net/python/pyproject | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1345452427 | I_kwDODLZ_YM5QMfmL | 11 | -a option is used for "--auth" and for "--all" | 2467 | closed | 0 | 3 | 2022-08-21T10:50:48Z | 2022-08-21T21:11:57Z | 2022-08-21T21:11:57Z | NONE | I'm not sure which option is best, instead of -a -all. | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1246826792 | I_kwDODLZ_YM5KUREo | 10 | When running `auth` command, don't overwrite an existing auth.json file | 11887 | closed | 0 | 3 | 2022-05-24T16:42:20Z | 2022-09-07T15:07:38Z | 2022-08-22T16:17:19Z | NONE | Ran the `auth` command in the same directory I'd previously set up an auth.json file for `twitter-to-sqlite` and it was completely overwritten. Not the biggest issue, but still unexpected. Ideally, for me, the keys would just be added to the existing file, but getting a warning and a chance to back out would be a good solution as well. | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
750141615 | MDExOlB1bGxSZXF1ZXN0NTI2ODQ3ODIz | 7 | Fixed conflicting CLI flags | 8944 | closed | 0 | 1 | 2020-11-24T23:25:12Z | 2022-08-21T21:11:56Z | 2022-08-21T21:11:56Z | CONTRIBUTOR | dogsheep/pocket-to-sqlite/pulls/7 | The `-a` used for the auth credentials and the shortened form of the `--all` flags were in conflict on the `fetch` command. To be consistent with other `-to-sqlite` libraries in the Dogsheep ecosystem, I removed the shortened form of the `--all` flag. | 213286752 | pull | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
589402939 | MDU6SXNzdWU1ODk0MDI5Mzk= | 4 | Store authentication information as "pocket_access_token" etc | 9599 | closed | 0 | 0 | 2020-03-27T20:43:22Z | 2020-03-27T20:43:59Z | 2020-03-27T20:43:59Z | MEMBER | The `pocket_` prefix will mean that the same `auth.json` file can be used for other Dogsheep tools without Pocket over-riding a value set by some other tool. | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
503234169 | MDU6SXNzdWU1MDMyMzQxNjk= | 2 | Track and use the 'since' value | 9599 | closed | 0 | 3 | 2019-10-07T05:02:59Z | 2020-03-27T22:22:30Z | 2020-03-27T22:22:30Z | MEMBER | Pocket says: > Whenever possible, you should use the since parameter, or count and and offset parameters when retrieving a user's list. After retrieving the list, you should store the current time (which is provided along with the list response) and pass that in the next request for the list. This way the server only needs to return a small set (changes since that time) instead of the user's entire list every time. At the bottom of https://getpocket.com/developer/docs/v3/retrieve | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/2/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
503233021 | MDU6SXNzdWU1MDMyMzMwMjE= | 1 | Use better pagination (and implement progress bar) | 9599 | closed | 0 | 4 | 2019-10-07T04:58:11Z | 2020-03-27T22:13:57Z | 2020-03-27T22:13:57Z | MEMBER | Right now we attempt to load everything at once - which caps out at 5,000 items and is really slow. We can do better by implementing pagination using count and offset. | 213286752 | issue | { "url": "https://api.github.com/repos/dogsheep/pocket-to-sqlite/issues/1/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1308461063 | I_kwDODFdgUs5N_YgH | 74 | 500 error in github-to-sqlite demo | 9599 | closed | 0 | 5 | 2022-07-18T19:39:32Z | 2022-07-18T21:16:18Z | 2022-07-18T21:14:22Z | MEMBER | https://github-to-sqlite.dogsheep.net/github/issue_comments throws a 500: > `cannot import name 'etree' from 'markdown.util' (/usr/local/lib/python3.8/site-packages/markdown/util.py)` https://console.cloud.google.com/run/detail/us-central1/github-to-sqlite/metrics?project=datasette-222320 suggests this started happening 3 days ago. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/74/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
1261884917 | PR_kwDODFdgUs45K1L3 | 73 | Fixing 'NoneType' object has no attribute 'items' | 1224205 | closed | 0 | 1 | 2022-06-06T13:58:11Z | 2022-07-18T19:40:12Z | 2022-07-18T19:40:12Z | CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/73 | Under some conditions, GitHub caches removed starred repositories and ends up leaving dangling `None` user references. Traceback (most recent call last): File "/home/dogsheep/dogsheep/github-to-sqlite/bin/github-to-sqlite", line 8, in <module> sys.exit(cli()) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/cli.py", line 181, in starred utils.save_stars(db, user, stars) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py", line 494, in save_stars repo_id = save_repo(db, repo) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py", line 308, in save_repo to_save["owner"] = save_user(db, to_save["owner"]) File "/home/dogsheep/dogsheep/github-to-sqlite/lib64/python3.10/site-packages/github_to_sqlite/utils.py", line 229, in save_user for key, value in user.items() AttributeError: 'NoneType' object has no attribute 'items' | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/73/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
1177059481 | I_kwDODFdgUs5GKICZ | 71 | Store commit parents | 64686 | closed | 0 | 0 | 2022-03-22T17:06:48Z | 2022-04-22T12:44:04Z | 2022-04-22T12:44:04Z | NONE | Hi @simonw 👋 Currently, stored commit data doesn't quite give me the information I'm needing... Committer date and author date are not 100% reliable for dividing a commit history up by release or branch. A PR created before a release but merged after can have earlier dates… — this can be quite frustrating if you're trying to pin down commits for a release: _It should be there!_, but then isn't. (This gets worse using release branches.) Would you be open to adding the `sha` of a `parent` of a commit to the commit table? (As an FK? 🤔 — likely not feasible.) It's part of the [response body](https://docs.github.com/en/rest/reference/commits#get-a-commit): ``` "parents": [ { "url": "https://api.github.com/repos/octocat/Hello-World/commits/6dcb09b5b57875f334f61aebed695e2e4193db5e", "sha": "6dcb09b5b57875f334f61aebed695e2e4193db5e" } ], ``` I think this list should only have a single entry. (🤔 — not sure why it's a list then...) With this it would be possible to build/reconstruct a chain of commits from the history, that I don't **think** is available as yet (unless you know a better way). It is certainly possible to get sequential lists of commits out of git directly, so the same would be possible combining tools, but wondering if a single tool could do it. What do you think? Thanks! 🏅 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/71/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
797108702 | MDExOlB1bGxSZXF1ZXN0NTY0MTcyMTQw | 61 | fixing typo in get cli help text | 22578954 | closed | 0 | 1 | 2021-01-29T18:57:04Z | 2021-05-19T16:07:09Z | 2021-05-19T16:07:09Z | CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/61 | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/61/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | ||||||
771872303 | MDExOlB1bGxSZXF1ZXN0NTQzMjQ2NTM1 | 59 | Remove unneeded exists=True for -a/--auth flag. | 631242 | closed | 0 | 3 | 2020-12-21T06:03:55Z | 2021-05-22T14:06:19Z | 2021-05-19T16:08:12Z | CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/59 | The file does not need to exist when using an environment variable. | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/59/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
769150394 | MDU6SXNzdWU3NjkxNTAzOTQ= | 58 | Readme HTML has broken internal links | 9599 | closed | 0 | 2 | 2020-12-16T17:58:11Z | 2020-12-16T19:20:14Z | 2020-12-16T19:20:14Z | MEMBER | From https://github.com/simonw/datasette.io/issues/46 ```html <li><a href="#filtering-tables">Filtering tables</a></li> ... <h3><a id="user-content-filtering-tables" class="anchor" aria-hidden="true" href="#filtering-tables"><svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M7.775 3.275a.75.75 0 001.06 1.06l1.25-1.25a2 2 0 112.83 2.83l-2.5 2.5a2 2 0 01-2.83 0 .75.75 0 00-1.06 1.06 3.5 3.5 0 004.95 0l2.5-2.5a3.5 3.5 0 00-4.95-4.95l-1.25 1.25zm-4.69 9.64a2 2 0 010-2.83l2.5-2.5a2 2 0 012.83 0 .75.75 0 001.06-1.06 3.5 3.5 0 00-4.95 0l-2.5 2.5a3.5 3.5 0 004.95 4.95l1.25-1.25a.75.75 0 00-1.06-1.06l-1.25 1.25a2 2 0 01-2.83 0z"></path></svg></a>Filtering tables</h3> ``` So this is a bug in GitHub's API, but we need to work around it. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/58/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
758944006 | MDU6SXNzdWU3NTg5NDQwMDY= | 57 | --readme throws 404 error if README does not exist in repo | 9599 | closed | 0 | 0 | 2020-12-07T23:58:49Z | 2020-12-16T18:17:54Z | 2020-12-16T18:17:54Z | MEMBER | It should fail silently (populate the column with a null) instead. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/57/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
753122082 | MDU6SXNzdWU3NTMxMjIwODI= | 56 | Link to example tables from the README | 9599 | closed | 0 | 0 | 2020-11-30T04:01:51Z | 2020-11-30T04:10:27Z | 2020-11-30T04:10:27Z | MEMBER | Would help demonstrate how the tool works. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/56/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
753026388 | MDU6SXNzdWU3NTMwMjYzODg= | 55 | github-to-sqlite workflows does not correctly replace existing records | 9599 | closed | 0 | 0 | 2020-11-29T21:58:43Z | 2020-11-29T23:48:50Z | 2020-11-29T23:48:50Z | MEMBER | Following #54 - see this TODO: https://github.com/dogsheep/github-to-sqlite/blob/1b23ce11953f9f59c0161ea1f99188b55b5ea11c/github_to_sqlite/utils.py#L700 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/55/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
753026003 | MDU6SXNzdWU3NTMwMjYwMDM= | 54 | github-to-sqlite workflows command | 9599 | closed | 0 | 3 | 2020-11-29T21:56:42Z | 2020-11-29T22:08:46Z | 2020-11-29T21:57:17Z | MEMBER | A command that fetches the YAML workflows for different repos, parses them and stores them in relational tables would be really useful for maintaining larger numbers of workflows. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/54/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
724264574 | MDU6SXNzdWU3MjQyNjQ1NzQ= | 52 | Option to fetch README and/or HTML-rendered README for repos | 9599 | closed | 0 | 0 | 2020-10-19T05:10:24Z | 2020-10-19T05:33:42Z | 2020-10-19T05:33:42Z | MEMBER | I'm thinking: github-to-sqlite repos ... --readme # Populates readme column with raw text github-to-sqlite repos ... --readme-html # Populates readme_html column with raw HTML https://developer.github.com/v3/repos/contents/#get-a-repository-readme | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/52/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
681228542 | MDExOlB1bGxSZXF1ZXN0NDY5NjUxNzMy | 48 | Add pull requests | 755825 | closed | 0 | 2 | 2020-08-18T17:58:44Z | 2020-11-29T23:51:09Z | 2020-11-29T23:51:09Z | CONTRIBUTOR | dogsheep/github-to-sqlite/pulls/48 | ref #46 Issues don't have merge information on them, which means that PRs need to be pulled separately. Did my best to mimic the API of issues. | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/48/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
681086659 | MDU6SXNzdWU2ODEwODY2NTk= | 47 | emojis command | 9599 | closed | 0 | 1 | 2020-08-18T14:26:26Z | 2020-08-18T14:52:13Z | 2020-08-18T14:52:13Z | MEMBER | For fun - it can import https://api.github.com/emojis - maybe with an option to fetch the binary representations in addition to the URLs. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/47/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
660429601 | MDU6SXNzdWU2NjA0Mjk2MDE= | 45 | Fix the demo - it breaks because of the tags table change | 9599 | closed | 0 | 5 | 2020-07-18T22:49:32Z | 2020-07-18T23:03:14Z | 2020-07-18T23:03:13Z | MEMBER | https://github.com/dogsheep/github-to-sqlite/runs/885773677 ``` File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 476, in save_tags db["tags"].insert_all( File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/sqlite_utils/db.py", line 1145, in insert_all result = self.db.conn.execute(query, params) sqlite3.OperationalError: table tags has no column named repo ``` That's because I changed the name in #44. I thought this would be safe since no-one else could possibly be using this yet (it hadn't shipped in a release) but turns out I broke my demo! | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/45/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
660413281 | MDU6SXNzdWU2NjA0MTMyODE= | 44 | Rename tags.repo_id column to tags.repo | 9599 | closed | 0 | 0 | 2020-07-18T22:13:46Z | 2020-07-18T22:15:12Z | 2020-07-18T22:15:12Z | MEMBER | For improved consistency with other tables. https://observablehq.com/@simonw/datasette-table-diagram ![datasette-table-diagram(1)](https://user-images.githubusercontent.com/9599/87862843-3cca4900-c909-11ea-9c76-58b3f4aca43f.png) | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/44/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
660355904 | MDU6SXNzdWU2NjAzNTU5MDQ= | 43 | github-to-sqlite tags command for fetching tags | 9599 | closed | 0 | 4 | 2020-07-18T20:14:12Z | 2020-07-18T23:05:56Z | 2020-07-18T21:52:15Z | MEMBER | Fetches paginated data from https://api.github.com/repos/simonw/datasette/tags | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/43/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
654405302 | MDU6SXNzdWU2NTQ0MDUzMDI= | 42 | Option for importing just specific repos | 9599 | closed | 0 | 0 | 2020-07-09T23:20:15Z | 2020-07-09T23:25:35Z | 2020-07-09T23:25:35Z | MEMBER | For if you know which specific repos you care about, as opposed to loading everything owned by the authenticated user. github-to-sqlite repos specific.db -r simonw/datasette -r simonw/github-contents | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/42/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
651159727 | MDU6SXNzdWU2NTExNTk3Mjc= | 41 | Demo is failing to deploy | 9599 | closed | 0 | 7 | 2020-07-05T22:40:33Z | 2020-07-06T01:07:03Z | 2020-07-06T01:07:02Z | MEMBER | https://github.com/dogsheep/github-to-sqlite/runs/837714622?check_suite_focus=true ``` Creating Revision.........................................................................................................................................failed Deployment failed ERROR: (gcloud.run.deploy) Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revision might contain more information. Traceback (most recent call last): File "/opt/hostedtoolcache/Python/3.8.3/x64/bin/datasette", line 8, in <module> sys.exit(cli()) File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/publish/cloudrun.py", line 138, in cloudrun check_call( File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/subprocess.py", line 364, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'gcloud run deploy --allow-unauthenticated --platform=managed --image gcr.io/datasette-222320/datasette github-to-sqlite' returned non-zero exit sta… | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/41/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
637899539 | MDU6SXNzdWU2Mzc4OTk1Mzk= | 40 | Demo deploy is broken | 9599 | closed | 0 | 2 | 2020-06-12T17:20:17Z | 2020-06-12T18:06:48Z | 2020-06-12T18:06:48Z | MEMBER | https://github.com/dogsheep/github-to-sqlite/runs/766180404?check_suite_focus=true ``` The following NEW packages will be installed: sqlite3 0 upgraded, 1 newly installed, 0 to remove and 11 not upgraded. Need to get 752 kB of archives. After this operation, 2482 kB of additional disk space will be used. Ign:1 http://azure.archive.ubuntu.com/ubuntu bionic-updates/main amd64 sqlite3 amd64 3.22.0-1ubuntu0.3 Err:1 http://security.ubuntu.com/ubuntu bionic-updates/main amd64 sqlite3 amd64 3.22.0-1ubuntu0.3 404 Not Found [IP: 52.177.174.250 80] E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/s/sqlite3/sqlite3_3.22.0-1ubuntu0.3_amd64.deb 404 Not Found [IP: 52.177.174.250 80] E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? ##[error]Process completed with exit code 100. ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
613777056 | MDU6SXNzdWU2MTM3NzcwNTY= | 39 | issues foreign key to repo isn't working | 9599 | closed | 0 | 1 | 2020-05-07T05:11:48Z | 2020-08-18T14:24:46Z | 2020-08-18T14:23:56Z | MEMBER | https://github-to-sqlite.dogsheep.net/github/issues?_facet=repo <img width="725" alt="github__issues__2_303_rows_where_sorted_by_updated_at_descending" src="https://user-images.githubusercontent.com/9599/81256810-723be780-8fe6-11ea-9e1f-f37d40f4d3ae.png"> If the foreign key was working those would be repository names. From the schema at the bottom of the page: ``` [repo] TEXT, ``` That's the wrong type and not a foreign key. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/39/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
611284481 | MDU6SXNzdWU2MTEyODQ0ODE= | 38 | [Feature Request] Support Repo Name in Search 🥺 | 5779832 | closed | 0 | 4 | 2020-05-02T22:08:51Z | 2020-05-03T02:34:32Z | 2020-05-02T23:15:11Z | NONE | ## Description Per your [v2.2 release tweet](https://twitter.com/simonw/status/1256700238099693568) I played with the demo, but the output did not match my expectations. ## Expected Behavior Expected a search query for "twitter" contained within the `repo` column to return non-zero results. ## Actual Behavior 😭 [0 rows where repo contains "twitter" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=twitter&_sort_desc=starred_at) ## Best Explanation Per the table schema (see appendix) `repo` is of type `INTEGER` which built from `repo_id` and does not expose the repo name in search. ## Desired Behavior Given that searching for "206156866" is less intuitive than "twitter", it would be great to support this via extending the search capabilities or by adding an additional column. ✅ 104 rows where repo contains "twitter" ❌ [104 rows where repo contains "206156866" sorted by starred_at descending](https://github-to-sqlite.dogsheep.net/github/stars?repo__contains=206156866&_sort_desc=starred_at) ## Appendix ``` CREATE TABLE [stars] ( [user] INTEGER REFERENCES [users]([id]), [repo] INTEGER REFERENCES [repos]([id]), [starred_at] TEXT, PRIMARY KEY ([user], [repo]) ); CREATE INDEX [idx_stars_repo] ON [stars] ([repo]); CREATE INDEX [idx_stars_user] ON [stars] ([user]); ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/38/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610843136 | MDU6SXNzdWU2MTA4NDMxMzY= | 37 | Mechanism for creating views if they don't yet exist | 9599 | closed | 0 | 3 | 2020-05-01T16:34:10Z | 2020-05-02T16:19:47Z | 2020-05-02T16:19:31Z | MEMBER | Needed for #36 #10 #12 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/37/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610842926 | MDU6SXNzdWU2MTA4NDI5MjY= | 36 | Add view for better display of dependent repos | 9599 | closed | 0 | 2 | 2020-05-01T16:33:44Z | 2020-05-02T16:50:31Z | 2020-05-02T16:30:11Z | MEMBER | ```sql select repos.full_name as repo, 'https://github.com/' || repos2.full_name as dependent, repos2.created_at as dependent_repo_created, repos2.updated_at as dependent_repo_updated, repos2.stargazers_count as dependent_repo_stars, repos2.watchers_count as dependent_repo_watchers from dependents join repos as repos2 on dependents.dependent = repos2.id join repos on dependents.repo = repos.id order by repos2.created_at desc ``` https://dogsheep.simonwillison.net/github?sql=select%0D%0A++repos.full_name+as+repo%2C%0D%0A++%27https%3A%2F%2Fgithub.com%2F%27+%7C%7C+repos2.full_name+as+dependent%2C%0D%0A++repos2.created_at+as+dependent_repo_created%2C%0D%0A++repos2.updated_at+as+dependent_repo_updated%2C%0D%0A++repos2.stargazers_count+as+dependent_repo_stars%2C%0D%0A++repos2.watchers_count+as+dependent_repo_watchers%0D%0Afrom%0D%0A++dependents%0D%0A++join+repos+as+repos2+on+dependents.dependent+%3D+repos2.id%0D%0A++join+repos+on+dependents.repo+%3D+repos.id%0D%0Aorder+by%0D%0A++repos2.created_at+desc | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/36/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610511450 | MDU6SXNzdWU2MTA1MTE0NTA= | 35 | Create index on issue_comments(user) and other foreign keys | 9599 | closed | 0 | 3 | 2020-05-01T02:06:56Z | 2020-05-02T18:26:24Z | 2020-05-02T18:26:24Z | MEMBER | ``` create index issue_comments_user on issue_comments(user) ``` I'm sure there are other user columns that could benefit from an index. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/35/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
610408908 | MDU6SXNzdWU2MTA0MDg5MDg= | 34 | Command for retrieving dependents for a repo | 9599 | closed | 0 | 6 | 2020-04-30T21:47:51Z | 2020-05-03T15:53:01Z | 2020-05-03T15:53:01Z | MEMBER | I really, really want to start grabbing this data: https://github.com/simonw/datasette/network/dependents | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/34/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
609950090 | MDU6SXNzdWU2MDk5NTAwOTA= | 33 | Fall back to authentication via ENV | 2029 | closed | 0 | 4 | 2020-04-30T12:58:14Z | 2020-05-02T18:46:10Z | 2020-05-02T18:45:37Z | NONE | Would you accept a PR that falls back to looking for an environment variable for the GitHub token? Specifically a change here: https://github.com/dogsheep/github-to-sqlite/blob/c34d5a18bfc41fa08755ba3d5cf9fe09ff204238/github_to_sqlite/cli.py#L271 I'd like to use `github-to-sqlite` in a GitHub Action workflow and this would be simpler than trying to fill out the prompt or generate a file with sensitive content. Wanted to check first, I'm happy to submit a PR with tests and updates to the docs. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/33/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
604222295 | MDU6SXNzdWU2MDQyMjIyOTU= | 32 | Issue comments don't appear to populate issues foreign key | 9599 | closed | 0 | 3 | 2020-04-21T19:17:32Z | 2020-04-22T01:17:44Z | 2020-04-22T01:17:44Z | MEMBER | https://github-to-sqlite.dogsheep.net/github?sql=select+html_url%2C+id%2C+issue+from+issue_comments+order+by+updated_at+desc+limit+101 <img width="774" alt="Screen Shot 2020-04-21 at 12 16 57 PM" src="https://user-images.githubusercontent.com/9599/79904752-0b091b00-83ca-11ea-9b15-b2893f05dfdc.png"> | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/32/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
603624862 | MDU6SXNzdWU2MDM2MjQ4NjI= | 31 | Issue and milestone should have foreign key to repo | 9599 | closed | 0 | 3 | 2020-04-21T00:46:24Z | 2020-04-22T01:20:19Z | 2020-04-22T01:20:19Z | MEMBER | Currently the `repo` column on those tables is a string `simonw/datasette` rather than an ID referencing a row in `repos`. _Originally posted by @simonw in https://github.com/dogsheep/github-to-sqlite/issues/29#issuecomment-616883275_ | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/31/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
603618244 | MDU6SXNzdWU2MDM2MTgyNDQ= | 30 | Issues milestone column is the wrong type | 9599 | closed | 0 | 2 | 2020-04-21T00:24:34Z | 2020-04-21T00:45:23Z | 2020-04-21T00:36:22Z | MEMBER | https://github-to-sqlite.dogsheep.net/github/issues?milestone=2857392 ![2A4C1185-2434-4F29-9EA0-3246E2F03F77](https://user-images.githubusercontent.com/9599/79811760-b7e08b00-832b-11ea-9ad7-684a6ae097a6.jpeg) It is TEXT when it should be an INTEGER - which is why the foreign key label is not correctly displayed. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/30/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
603617013 | MDU6SXNzdWU2MDM2MTcwMTM= | 29 | Milestones should have foreign key to creator and repo | 9599 | closed | 0 | 1 | 2020-04-21T00:20:44Z | 2020-04-21T00:43:58Z | 2020-04-21T00:43:58Z | MEMBER | https://github-to-sqlite.dogsheep.net/github/milestones Creator is an integer but not a foreign key to users Repo is missing entirely! | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/29/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
601333634 | MDU6SXNzdWU2MDEzMzM2MzQ= | 28 | Pull repository contributors | 9599 | closed | 0 | 3 | 2020-04-16T18:46:40Z | 2020-04-18T15:05:10Z | 2020-04-18T15:05:10Z | MEMBER | https://developer.github.com/v3/repos/#list-contributors `GET /repos/:owner/:repo/contributors` Not sure if this should be a separate command or should be part of the existing `repos` command. I'm leaning towards a new `contributors` command. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/28/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
601330277 | MDU6SXNzdWU2MDEzMzAyNzc= | 27 | Repos have a big blob of JSON in the organization column | 9599 | closed | 0 | 5 | 2020-04-16T18:43:14Z | 2020-04-18T00:19:16Z | 2020-04-18T00:18:52Z | MEMBER | e.g. https://github-to-sqlite.dogsheep.net/github/repos ![github__repos__11_rows_where_sorted_by_updated_at_descending](https://user-images.githubusercontent.com/9599/79494124-5640b980-7fd7-11ea-99a2-17ffbd82f9ce.png) This appears to be obsolete because the `owner` column already links to that record, albeit in the `users` table with `type` set to `Organization`: https://github-to-sqlite.dogsheep.net/github/users/53015001 | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/27/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
601271612 | MDU6SXNzdWU2MDEyNzE2MTI= | 26 | Topics are missing from repositories | 9599 | closed | 0 | 2 | 2020-04-16T17:36:32Z | 2020-04-16T17:41:11Z | 2020-04-16T17:41:11Z | MEMBER | I'm sure this used to work, but right now repositories are fetched without their topics. https://developer.github.com/v3/repos/ says you need to send a custom `Accept` header of `application/vnd.github.mercy-preview+json` to get topics. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/26/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
601265023 | MDU6SXNzdWU2MDEyNjUwMjM= | 25 | Improvements to demo instance | 9599 | closed | 0 | 1 | 2020-04-16T17:26:55Z | 2020-04-16T18:07:12Z | 2020-04-16T18:07:12Z | MEMBER | - [x] Demo should pull issue-comments as well | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/25/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
586595839 | MDU6SXNzdWU1ODY1OTU4Mzk= | 23 | Release 1.0 | 9599 | closed | 0 | 5225818 | 1 | 2020-03-24T00:03:55Z | 2020-03-24T00:15:50Z | 2020-03-24T00:15:50Z | MEMBER | Need to compile release notes. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/23/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
586567379 | MDU6SXNzdWU1ODY1NjczNzk= | 22 | Handle empty git repositories | 9599 | closed | 0 | 0 | 2020-03-23T22:49:48Z | 2020-03-23T23:13:11Z | 2020-03-23T23:13:11Z | MEMBER | Got this error: ``` github_to_sqlite.utils.GitHubError: {'message': 'Git Repository is empty.', 'documentation_url': 'https://developer.github.com/v3/repos/commits/#list-commits-on-a-repository'} ``` From https://api.github.com/repos/dogsheep/beta/commits | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/22/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
586561727 | MDU6SXNzdWU1ODY1NjE3Mjc= | 21 | Turn GitHub API errors into exceptions | 9599 | closed | 0 | 5225818 | 2 | 2020-03-23T22:37:24Z | 2020-03-23T23:48:23Z | 2020-03-23T23:48:22Z | MEMBER | This would have really helped in debugging the mess in #13. Running with this `auth.json` is a useful demo: ```json {"github_personal_token": ""} ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/21/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
586454513 | MDU6SXNzdWU1ODY0NTQ1MTM= | 20 | Upgrade to sqlite-utils 2.x | 9599 | closed | 0 | 5225818 | 0 | 2020-03-23T19:17:58Z | 2020-03-23T19:22:52Z | 2020-03-23T19:22:52Z | MEMBER | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/20/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
585850715 | MDU6SXNzdWU1ODU4NTA3MTU= | 19 | Enable full-text search for more stuff (like commits, issues and issue_comments) | 9599 | closed | 0 | 5225818 | 2 | 2020-03-23T00:19:56Z | 2020-03-23T19:06:39Z | 2020-03-23T19:06:39Z | MEMBER | Currently FTS is only enabled for repos and releases. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/19/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
585411547 | MDU6SXNzdWU1ODU0MTE1NDc= | 18 | Commits in GitHub API can have null author | 9599 | closed | 0 | 5225818 | 8 | 2020-03-21T02:20:56Z | 2020-03-23T20:44:49Z | 2020-03-23T20:44:26Z | MEMBER | ``` Traceback (most recent call last): File "/home/ubuntu/datasette-venv/bin/github-to-sqlite", line 8, in <module> sys.exit(cli()) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/github_to_sqlite/cli.py", line 235, in commits utils.save_commits(db, commits, repo_full["id"]) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/github_to_sqlite/utils.py", line 290, in save_commits commit_to_insert["author"] = save_user(db, commit["author"]) File "/home/ubuntu/datasette-venv/lib/python3.6/site-packages/github_to_sqlite/utils.py", line 54, in save_user for key, value in user.items() AttributeError: 'NoneType' object has no attribute 'items' ``` Got this running the `commits` command from cron. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/18/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
578883725 | MDU6SXNzdWU1Nzg4ODM3MjU= | 17 | Command for importing commits | 9599 | closed | 0 | 2 | 2020-03-10T21:55:12Z | 2020-03-11T02:47:37Z | 2020-03-11T02:47:37Z | MEMBER | Using this API: https://api.github.com/repos/dogsheep/github-to-sqlite/commits | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/17/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
546051181 | MDU6SXNzdWU1NDYwNTExODE= | 16 | Exception running first command: IndexError: list index out of range | 15092 | closed | 0 | 4 | 2020-01-07T03:01:58Z | 2020-04-14T18:37:21Z | 2020-04-14T18:37:21Z | NONE | Exception running first command without an existing db or auth. ```py > mkdir ~/.github/coala > /usr/bin/github-to-sqlite repos ~/.github/coala coala Traceback (most recent call last): File "/usr/bin/github-to-sqlite", line 11, in <module> load_entry_point('github-to-sqlite==0.6', 'console_scripts', 'github-to-sqlite')() File "/usr/lib/python3.7/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/usr/lib/python3.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/usr/lib/python3.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/lib/python3.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/lib/python3.7/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/usr/lib/python3.7/site-packages/github_to_sqlite/cli.py", line 163, in repos utils.save_repo(db, repo) File "/usr/lib/python3.7/site-packages/github_to_sqlite/utils.py", line 120, in save_repo to_save["owner"] = save_user(db, to_save["owner"]) File "/usr/lib/python3.7/site-packages/github_to_sqlite/utils.py", line 61, in save_user return db["users"].upsert(to_save, pk="id", alter=True).last_pk File "/usr/lib/python3.7/site-packages/sqlite_utils/db.py", line 1135, in upsert extracts=extracts, File "/usr/lib/python3.7/site-packages/sqlite_utils/db.py", line 1162, in upsert_all upsert=True, File "/usr/lib/python3.7/site-packages/sqlite_utils/db.py", line 1105, in insert_all row = list(self.rows_where("rowid = ?", [self.last_rowid]))[0] IndexError: list index out of range ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/16/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
544571092 | MDU6SXNzdWU1NDQ1NzEwOTI= | 15 | Assets table with downloads | 2029 | closed | 0 | 5225818 | 4 | 2020-01-02T13:05:28Z | 2020-03-28T12:17:01Z | 2020-03-23T19:17:32Z | NONE | The `releases` command extracts the releases table, but data about the individual assets are locked up in the JSON document in the `assets` field. My main interest is in individual and aggregate download counts. I was wondering if creating a new table with a record per asset may be useful? If so I'm happy to send a PR when I get a moment. Do you have opinions about that simply being part of the `releases` command or would you prefer a separate command as well? | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/15/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
521275281 | MDU6SXNzdWU1MjEyNzUyODE= | 13 | Set up a live demo Datasette instance | 9599 | closed | 0 | 5225818 | 9 | 2019-11-12T01:27:02Z | 2020-03-24T00:03:26Z | 2020-03-24T00:03:25Z | MEMBER | I deployed https://github-to-sqlite-releases-j7hipcg4aq-uc.a.run.app/ by running this: ``` #!/bin/bash # Fetch repos for simonw and dogsheep github-to-sqlite repos github.db simonw dogsheep -a auth.json # Fetch releases for the repos tagged 'datasette-io' sqlite-utils github.db " select full_name from repos where rowid in ( select repos.rowid from repos, json_each(repos.topics) j where j.value = 'datasette-io' )" --csv --no-headers | while read repo; do github-to-sqlite releases \ github.db $(echo $repo | tr -d '\r') \ -a auth.json; sleep 2; done; ``` And then deploying using this: ``` $ datasette publish cloudrun github.db \ --title "github-to-sqlite releases demo" \ --about_url="https://github.com/simonw/github-to-sqlite" \ --about='github-to-sqlite' \ --install=datasette-render-markdown \ --install=datasette-json-html \ --service=github-to-sqlite-releases ``` This should happen automatically for every release. I can run it once a day in Circle CI to keep the demo database up-to-date. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/13/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | |||||
520756546 | MDU6SXNzdWU1MjA3NTY1NDY= | 12 | Add this view for seeing new releases | 9599 | closed | 0 | 5 | 2019-11-11T06:00:12Z | 2020-05-02T18:58:18Z | 2020-05-02T18:58:17Z | MEMBER | ```sql CREATE VIEW recent_releases AS select json_object("label", repos.full_name, "href", repos.html_url) as repo, json_object( "href", releases.html_url, "label", releases.name ) as release, substr(releases.published_at, 0, 11) as date, releases.body as body_markdown, releases.published_at from releases join repos on repos.id = releases.repo order by releases.published_at desc ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/12/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
520521843 | MDU6SXNzdWU1MjA1MjE4NDM= | 11 | Command to fetch releases | 9599 | closed | 0 | 0 | 2019-11-09T22:23:30Z | 2019-11-09T22:57:00Z | 2019-11-09T22:57:00Z | MEMBER | https://developer.github.com/v3/repos/releases/#list-releases-for-a-repository `GET /repos/:owner/:repo/releases` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/11/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
516967682 | MDU6SXNzdWU1MTY5Njc2ODI= | 10 | Add this repos_starred view | 9599 | closed | 0 | 3 | 2019-11-04T05:44:38Z | 2020-05-02T16:37:36Z | 2020-05-02T16:37:36Z | MEMBER | ```sql create view repos_starred as select stars.starred_at, users.login, repos.* from repos join stars on repos.id = stars.repo join users on repos.owner = users.id order by starred_at desc; ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/10/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
516769276 | MDU6SXNzdWU1MTY3NjkyNzY= | 9 | Commands do not work without an auth.json file | 9599 | closed | 0 | 0 | 2019-11-03T01:54:28Z | 2019-11-11T05:30:48Z | 2019-11-11T05:30:48Z | MEMBER | `auth.json` is meant to be optional. If it's not provided, the tool should make heavily rate-limited unauthenticated requests. ``` $ github-to-sqlite repos .data/repos.db simonw Usage: github-to-sqlite repos [OPTIONS] DB_PATH [USERNAME] Try "github-to-sqlite repos --help" for help. Error: Invalid value for "-a" / "--auth": File "auth.json" does not exist. ``` | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/9/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
516763727 | MDExOlB1bGxSZXF1ZXN0MzM1OTgwMjQ2 | 8 | stargazers command, refs #4 | 9599 | closed | 0 | 5 | 2019-11-03T00:37:36Z | 2020-05-02T20:00:27Z | 2020-05-02T20:00:26Z | MEMBER | dogsheep/github-to-sqlite/pulls/8 | Needs tests. Refs #4. | 207052882 | pull | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/8/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
0 | |||||
506276893 | MDU6SXNzdWU1MDYyNzY4OTM= | 7 | issue-comments command for importing issue comments | 9599 | closed | 0 | 1 | 2019-10-13T05:23:58Z | 2019-10-14T14:44:12Z | 2019-10-13T05:24:30Z | MEMBER | Using this API: https://developer.github.com/v3/issues/comments/ | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/7/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
504238461 | MDU6SXNzdWU1MDQyMzg0NjE= | 6 | sqlite3.OperationalError: table users has no column named bio | 1055831 | closed | 0 | 2 | 2019-10-08T19:39:52Z | 2019-10-13T05:31:28Z | 2019-10-13T05:30:19Z | NONE | ``` $ github-to-sqlite repos github.db $ github-to-sqlite starred github.db dazzag24 Traceback (most recent call last): File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/bin/github-to-sqlite", line 10, in <module> sys.exit(cli()) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/click/core.py", line 764, in __call__ return self.main(*args, **kwargs) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/github_to_sqlite/cli.py", line 106, in starred utils.save_stars(db, user, stars) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/github_to_sqlite/utils.py", line 177, in save_stars user_id = save_user(db, user) File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/github_to_sqlite/utils.py", line 61, in save_user return db["users"].upsert(to_save, pk="id").last_pk File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/sqlite_utils/db.py", line 1067, in upsert extracts=extracts, File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/sqlite_utils/db.py", line 916, in insert extracts=extracts, File "/home/darreng/.virtualenvs/dogsheep-d2PjdrD7/lib/python3.6/site-packages/sqlite_utils/db.py", line 1024, in insert_all result = self.db.conn.execute(sql, values)… | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/6/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
493671014 | MDU6SXNzdWU0OTM2NzEwMTQ= | 5 | Add "incomplete" boolean to users table for incomplete profiles | 9599 | closed | 0 | 2 | 2019-09-14T22:01:50Z | 2020-03-23T19:23:31Z | 2020-03-23T19:23:30Z | MEMBER | User profiles that are fetched from e.g. stargazers (#4) are incomplete - they have a login but they don't have name, company etc. Add a `incomplete` boolean flag to the `users` table to record this. Then later I can add a `backfill-users` command which loops through and fetches missing data for those incomplete profiles. | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/5/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
493670730 | MDU6SXNzdWU0OTM2NzA3MzA= | 4 | Command to fetch stargazers for one or more repos | 9599 | closed | 0 | 8 | 2019-09-14T21:58:22Z | 2020-05-02T21:30:27Z | 2020-05-02T21:30:27Z | MEMBER | Maybe this: $ github-to-sqlite stargazers github.db simonw/datasette It could accept more than one repos. Maybe have options similar to `--sql` in [twitter-to-sqlite](https://github.com/dogsheep/twitter-to-sqlite) so you can e.g. fetch all stargazers for all of the repos you have fetched into the database already (or all of the repos belonging to owner X) | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/4/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed | ||||||
493670426 | MDU6SXNzdWU0OTM2NzA0MjY= | 3 | Command to fetch all repos belonging to a user or organization | 9599 | closed | 0 | 2 | 2019-09-14T21:54:21Z | 2019-09-17T00:17:53Z | 2019-09-17T00:17:53Z | MEMBER | How about this: $ github-to-sqlite repos simonw | 207052882 | issue | { "url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/3/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
completed |