{"id": 1662951875, "node_id": "I_kwDOBm6k_c5jHqHD", "number": 2057, "title": "DeprecationWarning: pkg_resources is deprecated as an API", "user": {"value": 9599, "label": "simonw"}, "state": "closed", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2023-04-11T17:41:20Z", "updated_at": "2023-09-21T22:09:10Z", "closed_at": "2023-09-21T22:09:10Z", "author_association": "OWNER", "pull_request": null, "body": "Got this running tests against Python 3.11.\r\n\r\n```\r\n../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/datasette/app.py:14: in <module>\r\n    import pkg_resources\r\n../../../.local/share/virtualenvs/datasette-big-local-6Yn-280V/lib/python3.11/site-packages/pkg_resources/__init__.py:121: in <module>\r\n    warnings.warn(\"pkg_resources is deprecated as an API\", DeprecationWarning)\r\nE   DeprecationWarning: pkg_resources is deprecated as an API\r\n```\r\n\r\nI ran with `pytest -Werror --pdb -x` to get the debugger for that warning, but it turned out searching the code worked better. It's used in these two places:\r\n\r\nhttps://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/plugins.py#L43-L50\r\n\r\nhttps://github.com/simonw/datasette/blob/5890a20c374fb0812d88c9b0ef26a838bfa06c76/datasette/app.py#L1037", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/2057/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": "completed"}
{"id": 1124731464, "node_id": "I_kwDOCGYnMM5DCgpI", "number": 399, "title": "Make it easier to insert geometries, with documentation and maybe code", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2022-02-05T00:11:26Z", "updated_at": "2023-05-16T03:11:52Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "In playing with the new SpatiaLite helpers from #385 I noticed that actually populating geometry columns is still a little bit tricky. Here's what I ended up doing:\r\n\r\n```python\r\nimport httpx, sqlite_utils\r\ndb = sqlite_utils.Database(\"/tmp/spatial.db\")\r\nattractions = httpx.get(\"https://latest.datasette.io/fixtures/roadside_attractions.json?_shape=array\").json()\r\ndb[\"attractions\"].insert_all(attractions, pk=\"pk\")\r\n\r\n# Schema of that table is now:\r\n# CREATE TABLE [attractions] (\r\n#    [pk] INTEGER PRIMARY KEY,\r\n#    [name] TEXT,\r\n#    [address] TEXT,\r\n#    [latitude] FLOAT,\r\n#    [longitude] FLOAT\r\n# )\r\n\r\ndb.init_spatialite()\r\ndb[\"attractions\"].add_geometry_column(\"point\", \"POINT\")\r\n\r\ndb.execute(\"\"\"\r\n    update attractions set point = GeomFromText(\r\n      'POINT(' || longitude || ' ' || latitude || ')', 4326\r\n    )\r\n\"\"\")\r\n```\r\nThat last line took some figuring out - especially the need for the SRID of `4326`, without which I got this error:\r\n\r\n> `IntegrityError: attractions.point violates Geometry constraint [geom-type or SRID not allowed]`\r\n\r\nIt would be good to both document this in more detail, but ideally also to come up with a more obvious pattern for inserting common types of spatial data.\r\n\r\nAlso related:\r\n- #398\r\n- #79", "repo": {"value": 140912432, "label": "sqlite-utils"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/sqlite-utils/issues/399/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 639993467, "node_id": "MDU6SXNzdWU2Mzk5OTM0Njc=", "number": 850, "title": "Proof of concept for Datasette on AWS Lambda with EFS", "user": {"value": 9599, "label": "simonw"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2020-06-16T21:48:31Z", "updated_at": "2020-06-16T23:52:16Z", "closed_at": null, "author_association": "OWNER", "pull_request": null, "body": "https://aws.amazon.com/about-aws/whats-new/2020/06/aws-lambda-support-for-amazon-elastic-file-system-now-generally-/\r\n\r\nIf Datasette can run on Lambda with access to EFS it could both read AND write large databases there.", "repo": {"value": 107914493, "label": "datasette"}, "type": "issue", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/simonw/datasette/issues/850/reactions\", \"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": null, "state_reason": null}
{"id": 813880401, "node_id": "MDExOlB1bGxSZXF1ZXN0NTc3OTUzNzI3", "number": 5, "title": "WIP: Add Gmail takeout mbox import", "user": {"value": 306240, "label": "UtahDave"}, "state": "open", "locked": 0, "assignee": null, "milestone": null, "comments": 25, "created_at": "2021-02-22T21:30:40Z", "updated_at": "2021-07-28T07:18:56Z", "closed_at": null, "author_association": "FIRST_TIME_CONTRIBUTOR", "pull_request": "dogsheep/google-takeout-to-sqlite/pulls/5", "body": "WIP\r\n\r\nThis PR adds the ability to import emails from a Gmail mbox export from Google Takeout.\r\n\r\nThis is my first PR to a datasette/dogsheep repo. I've tested this on my personal Google Takeout mbox with ~520,000 emails going back to 2004. This took around ~20 minutes to process.\r\n\r\nTo provide some feedback on the progress of the import I added the \"rich\" python module. I'm happy to remove that if adding a dependency is discouraged. However, I think it makes a nice addition to give feedback on the progress of a long import.\r\n\r\nDo we want to log emails that have errors when trying to import them?\r\n\r\nDealing with encodings with emails is a bit tricky. I'm very open to feedback on how to deal with those better. As well as any other feedback for improvements.", "repo": {"value": 206649770, "label": "google-takeout-to-sqlite"}, "type": "pull", "active_lock_reason": null, "performed_via_github_app": null, "reactions": "{\"url\": \"https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/5/reactions\", \"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "draft": 0, "state_reason": null}