{"html_url": "https://github.com/simonw/datasette/issues/1175#issuecomment-762488336", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1175", "id": 762488336, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjQ4ODMzNg==", "user": {"value": 758858, "label": "hannseman"}, "created_at": "2021-01-18T21:59:28Z", "updated_at": "2021-01-18T22:00:31Z", "author_association": "NONE", "body": "I encountered your issue when trying to find a solution and came up with the following, maybe it can help.\r\n\r\n```python\r\nimport logging.config\r\nfrom typing import Tuple\r\n\r\nimport structlog\r\nimport uvicorn\r\n\r\nfrom example.config import settings\r\n\r\nshared_processors: Tuple[structlog.types.Processor, ...] = (\r\n    structlog.contextvars.merge_contextvars,\r\n    structlog.stdlib.add_logger_name,\r\n    structlog.stdlib.add_log_level,\r\n    structlog.processors.TimeStamper(fmt=\"iso\"),\r\n)\r\n\r\nlogging_config = {\r\n    \"version\": 1,\r\n    \"disable_existing_loggers\": False,\r\n    \"formatters\": {\r\n        \"json\": {\r\n            \"()\": structlog.stdlib.ProcessorFormatter,\r\n            \"processor\": structlog.processors.JSONRenderer(),\r\n            \"foreign_pre_chain\": shared_processors,\r\n        },\r\n        \"console\": {\r\n            \"()\": structlog.stdlib.ProcessorFormatter,\r\n            \"processor\": structlog.dev.ConsoleRenderer(),\r\n            \"foreign_pre_chain\": shared_processors,\r\n        },\r\n        **uvicorn.config.LOGGING_CONFIG[\"formatters\"],\r\n    },\r\n    \"handlers\": {\r\n        \"default\": {\r\n            \"level\": \"DEBUG\",\r\n            \"class\": \"logging.StreamHandler\",\r\n            \"formatter\": \"json\" if not settings.debug else \"console\",\r\n        },\r\n        \"uvicorn.access\": {\r\n            \"level\": \"INFO\",\r\n            \"class\": \"logging.StreamHandler\",\r\n            \"formatter\": \"access\",\r\n        },\r\n        \"uvicorn.default\": {\r\n            \"level\": \"INFO\",\r\n            \"class\": \"logging.StreamHandler\",\r\n            \"formatter\": \"default\",\r\n        },\r\n    },\r\n    \"loggers\": {\r\n        \"\": {\"handlers\": [\"default\"], \"level\": \"INFO\"},\r\n        \"uvicorn.error\": {\r\n            \"handlers\": [\"default\" if not settings.debug else \"uvicorn.default\"],\r\n            \"level\": \"INFO\",\r\n            \"propagate\": False,\r\n        },\r\n        \"uvicorn.access\": {\r\n            \"handlers\": [\"default\" if not settings.debug else \"uvicorn.access\"],\r\n            \"level\": \"INFO\",\r\n            \"propagate\": False,\r\n        },\r\n    },\r\n}\r\n\r\n\r\ndef setup_logging() -> None:\r\n    structlog.configure(\r\n        processors=[\r\n            structlog.stdlib.filter_by_level,\r\n            *shared_processors,\r\n            structlog.stdlib.PositionalArgumentsFormatter(),\r\n            structlog.processors.StackInfoRenderer(),\r\n            structlog.processors.format_exc_info,\r\n            structlog.stdlib.ProcessorFormatter.wrap_for_formatter,\r\n        ],\r\n        context_class=dict,\r\n        logger_factory=structlog.stdlib.LoggerFactory(),\r\n        wrapper_class=structlog.stdlib.AsyncBoundLogger,\r\n        cache_logger_on_first_use=True,\r\n    )\r\n    logging.config.dictConfig(logging_config)\r\n```\r\n\r\nAnd then it'll be run on the startup event:\r\n```python\r\n@app.on_event(\"startup\")\r\nasync def startup_event() -> None:\r\n    setup_logging()\r\n```\r\n\r\nIt depends on a setting called `debug` which controls if it should output the regular uvicorn logging or json. ", "reactions": "{\"total_count\": 15, \"+1\": 7, \"-1\": 0, \"laugh\": 1, \"hooray\": 1, \"confused\": 0, \"heart\": 5, \"rocket\": 1, \"eyes\": 0}", "issue": {"value": 779156520, "label": "Use structlog for logging"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1036#issuecomment-762391426", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1036", "id": 762391426, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM5MTQyNg==", "user": {"value": 4997607, "label": "philshem"}, "created_at": "2021-01-18T17:45:00Z", "updated_at": "2021-01-18T17:45:00Z", "author_association": "NONE", "body": "It might be possible with this library: https://docs.python.org/3/library/imghdr.html\r\n\r\nquick test of the downloaded blob:\r\n\r\n```\r\n>>> import imghdr\r\n>>> imghdr.what('material_culture-1-image.blob')\r\n'jpeg'\r\n```\r\n\r\nThe output plugin would be cool. I'll look into making my first datasette plugin.  I'm also imagining displaying the image in the browser -- but that would be a step 2.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 725996507, "label": "Make it possible to download BLOB data from the Datasette UI"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1194#issuecomment-762390568", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1194", "id": 762390568, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM5MDU2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T17:43:03Z", "updated_at": "2021-01-18T17:43:03Z", "author_association": "OWNER", "body": "Should I just blanket copy over any query string argument that starts with an underscore? Any reason _not_ to do that?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 788447787, "label": "?_size= argument is not persisted by hidden form fields in the table filters"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1194#issuecomment-762390401", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1194", "id": 762390401, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM5MDQwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T17:42:38Z", "updated_at": "2021-01-18T17:42:38Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/a882d679626438ba0d809944f06f239bcba8ee96/datasette/views/table.py#L815-L827\r\n\r\nIt looks like there are other arguments that may not be persisted too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 788447787, "label": "?_size= argument is not persisted by hidden form fields in the table filters"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1036#issuecomment-762387875", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1036", "id": 762387875, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM4Nzg3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T17:36:36Z", "updated_at": "2021-01-18T17:36:36Z", "author_association": "OWNER", "body": "As you can see, I'm pretty paranoid about serving content with `Content-Type` HTTP headers because I'm so worried about execution vulnerabilities. I'm much more comfortable exploring that kind of thing in plugins, since that way people can opt-in to riskier features.\r\n\r\nYou found `datasette-media` which is my most comprehensive exploration of that idea so far - but there's definitely lots of room for more plugins along those lines.\r\n\r\nMaybe even an output plugin? `.jpg` as an export format which returns the `BLOB` column for a row as a JPEG image with the correct `content-type` header (but first verifies that the binary content does indeed look like a real JPEG) could be interesting.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 725996507, "label": "Make it possible to download BLOB data from the Datasette UI"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1036#issuecomment-762385981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1036", "id": 762385981, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM4NTk4MQ==", "user": {"value": 4997607, "label": "philshem"}, "created_at": "2021-01-18T17:32:13Z", "updated_at": "2021-01-18T17:34:50Z", "author_association": "NONE", "body": "Hi Simon\r\n\r\nJust finding this old issue regarding downloading blobs. Nice work!\r\n\r\n<img width=\"212\" alt=\"image\" src=\"https://user-images.githubusercontent.com/4997607/104946741-df4bad80-59ba-11eb-96e3-727c85cc4dc6.png\">\r\n\r\nAs a feature request, maybe it would be possible to assign a blob column as a certain data type (e.g. `.jpg`) and then each blob could be downloaded as that type of file (perhaps if the file types were constrained to normal blobs that people store in sqlite databases, this could avoid the execution stuff mentioned above).\r\n\r\nI guess the column blob-type definition could fit into this dropdown selection:\r\n\r\n<img width=\"201\" alt=\"image\" src=\"https://user-images.githubusercontent.com/4997607/104947000-479a8f00-59bb-11eb-87d9-1644e5940894.png\">\r\n\r\nLet me know if I should open a new issue with a feature request. (This could slowly go in the direction of displaying image blob-types in the browser.)\r\n\r\nThanks for the great tool!\r\n\r\n\r\n---\r\n\r\nedit: just reading the rest of the twitter thread: https://twitter.com/simonw/status/1318685933256855552\r\n\r\nperhaps this is already possible in some form with the plugin datasette-media: https://github.com/simonw/datasette-media", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 725996507, "label": "Make it possible to download BLOB data from the Datasette UI"}, "performed_via_github_app": null}
{"html_url": "https://github.com/dogsheep/swarm-to-sqlite/issues/11#issuecomment-761967094", "issue_url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11", "id": 761967094, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTk2NzA5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T04:11:13Z", "updated_at": "2021-01-18T04:11:13Z", "author_association": "MEMBER", "body": "I just got a similar error:\r\n\r\n```\r\n  File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/swarm_to_sqlite/utils.py\", line 79, in save_checkin\r\n    checkins_table.m2m(\"users\", user, m2m_table=\"with\", pk=\"id\")\r\n  File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 2048, in m2m\r\n    id = other_table.insert(record, pk=pk, replace=True).last_pk\r\n  File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1781, in insert\r\n    return self.insert_all(\r\n  File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1899, in insert_all\r\n    self.insert_chunk(\r\n  File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1709, in insert_chunk\r\n    result = self.db.execute(query, params)\r\n  File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 226, in execute\r\n    return self.conn.execute(sql, parameters)\r\npysqlite3.dbapi2.OperationalError: table users has no column named countryCode\r\n\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 743400216, "label": "Error thrown: sqlite3.OperationalError: table users has no column named lastName"}, "performed_via_github_app": null}