{"html_url": "https://github.com/simonw/datasette/issues/1199#issuecomment-766181628", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1199", "id": 766181628, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NjE4MTYyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-23T21:25:18Z", "updated_at": "2021-01-23T21:25:18Z", "author_association": "OWNER", "body": "Comment thread here: https://news.ycombinator.com/item?id=25881911 - cperciva says:\r\n\r\n> There's an even better reason for databases to not write to memory mapped pages: Pages get synched out to disk at the kernel's leisure. This can be ok for a cache but it's definitely not what you want for a database!\r\n\r\nBut... Datasette is often used in read-only mode, so that disadvantage often doesn't apply.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 792652391, "label": "Experiment with PRAGMA mmap_size=N"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/pull/224#issuecomment-765678057", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/224", "id": 765678057, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTY3ODA1Nw==", "user": {"value": 37962604, "label": "polyrand"}, "created_at": "2021-01-22T20:53:06Z", "updated_at": "2021-01-23T20:13:27Z", "author_association": "NONE", "body": "I'm using the FTS methods in sqlite-utils for this website: [drwn.io](https://drwn.io/). I wanted to get pagination to have some kind of infinite scrolling in the landing page, and I ended up using that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 792297010, "label": "Add fts offset docs."}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1191#issuecomment-765757433", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1191", "id": 765757433, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTc1NzQzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-22T23:43:43Z", "updated_at": "2021-01-22T23:43:43Z", "author_association": "OWNER", "body": "Another potential use for this: plugins that provide authentication (like `datasette-auth-passwords` and `datasette-auth-github`) could use it to add a chunk of HTML to the \"permission denied\" page that links to their mechanism of authenticating.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 787098345, "label": "Ability for plugins to collaborate when adding extra HTML to blocks in default templates"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1196#issuecomment-765639968", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1196", "id": 765639968, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTYzOTk2OA==", "user": {"value": 2826376, "label": "QAInsights"}, "created_at": "2021-01-22T19:37:15Z", "updated_at": "2021-01-22T19:37:15Z", "author_association": "NONE", "body": "I tried deployment in WSL. It is working fine https://jmeter.vercel.app/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 791237799, "label": "Access Denied Error in Windows"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765525338", "issue_url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1", "id": 765525338, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTUyNTMzOA==", "user": {"value": 25372415, "label": "cobiadigital"}, "created_at": "2021-01-22T16:22:44Z", "updated_at": "2021-01-22T16:22:44Z", "author_association": "NONE", "body": "rs1333049 associated with coronary artery disease\r\nhttps://www.snpedia.com/index.php/Rs1333049\r\n```\r\n\r\nselect rsid, genotype, case genotype\r\n when 'CC' then '1.9x increased risk for coronary artery disease'\r\n when 'CG' then '1.5x increased risk for CAD'\r\n when 'GG' then 'normal'\r\nend as interpretation from genome where rsid = 'rs1333049'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 496415321, "label": "Figure out some interesting example SQL queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765523517", "issue_url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1", "id": 765523517, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTUyMzUxNw==", "user": {"value": 25372415, "label": "cobiadigital"}, "created_at": "2021-01-22T16:20:25Z", "updated_at": "2021-01-22T16:20:25Z", "author_association": "NONE", "body": "rs53576: the oxytocin receptor (OXTR) gene\r\n\r\n```\r\nselect rsid, genotype, case genotype\r\n when 'AA' then 'Lack of empathy?'\r\n when 'AG' then 'Lack of empathy?'\r\n when 'GG' then 'Optimistic and empathetic; handle stress well'\r\nend as interpretation from genome where rsid = 'rs53576'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 496415321, "label": "Figure out some interesting example SQL queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765506901", "issue_url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1", "id": 765506901, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTUwNjkwMQ==", "user": {"value": 25372415, "label": "cobiadigital"}, "created_at": "2021-01-22T15:58:41Z", "updated_at": "2021-01-22T15:58:58Z", "author_association": "NONE", "body": "Both rs10757274 and rs2383206 can both indicate higher risks of heart disease\r\nhttps://www.snpedia.com/index.php/Rs2383206\r\n\r\n```\r\nselect rsid, genotype, case genotype\r\n when 'AA' then 'Normal'\r\n when 'AG' then '~1.2x increased risk for heart disease'\r\n when 'GG' then '~1.3x increased risk for heart disease'\r\nend as interpretation from genome where rsid = 'rs10757274'\r\n```\r\n\r\n```\r\nselect rsid, genotype, case genotype\r\n when 'AA' then 'Normal'\r\n when 'AG' then '1.4x increased risk for heart disease'\r\n when 'GG' then '1.7x increased risk for heart disease'\r\nend as interpretation from genome where rsid = 'rs2383206'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 496415321, "label": "Figure out some interesting example SQL queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765502845", "issue_url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1", "id": 765502845, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTUwMjg0NQ==", "user": {"value": 25372415, "label": "cobiadigital"}, "created_at": "2021-01-22T15:53:19Z", "updated_at": "2021-01-22T15:53:19Z", "author_association": "NONE", "body": "rs7903146 Influences risk of Type-2 diabetes\r\nhttps://www.snpedia.com/index.php/Rs7903146\r\n```\r\nselect rsid, genotype, case genotype\r\n when 'CC' then 'Normal (lower) risk of Type 2 Diabetes and Gestational Diabetes.'\r\n when 'CT' then '1.4x increased risk for diabetes (and perhaps colon cancer).'\r\n when 'TT' then '2x increased risk for Type-2 diabetes'\r\nend as interpretation from genome where rsid = 'rs7903146'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 496415321, "label": "Figure out some interesting example SQL queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765498984", "issue_url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1", "id": 765498984, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTQ5ODk4NA==", "user": {"value": 25372415, "label": "cobiadigital"}, "created_at": "2021-01-22T15:48:25Z", "updated_at": "2021-01-22T15:49:33Z", "author_association": "NONE", "body": "The \"Warrior Gene\" https://www.snpedia.com/index.php/Rs4680\r\n\r\n```\r\nselect rsid, genotype, case genotype\r\n when 'AA' then '(worrier) advantage in memory and attention tasks'\r\n when 'AG' then 'Intermediate dopamine levels, other effects'\r\n when 'GG' then '(warrior) multiple associations, see details'\r\nend as interpretation from genome where rsid = 'rs4680'\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 496415321, "label": "Figure out some interesting example SQL queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/genome-to-sqlite/issues/1#issuecomment-765495861", "issue_url": "https://api.github.com/repos/dogsheep/genome-to-sqlite/issues/1", "id": 765495861, "node_id": "MDEyOklzc3VlQ29tbWVudDc2NTQ5NTg2MQ==", "user": {"value": 25372415, "label": "cobiadigital"}, "created_at": "2021-01-22T15:44:00Z", "updated_at": "2021-01-22T15:44:00Z", "author_association": "NONE", "body": "Risk of autoimmune disorders: https://www.snpedia.com/index.php/Genotype\r\n```\r\nselect rsid, genotype, case genotype\r\n when 'AA' then '2x risk of rheumatoid arthritis and other autoimmune diseases'\r\n when 'GG' then 'Normal risk for autoimmune disorders'\r\nend as interpretation from genome where rsid = 'rs2476601'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 496415321, "label": "Figure out some interesting example SQL queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1195#issuecomment-763108730", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1195", "id": 763108730, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MzEwODczMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-19T20:22:37Z", "updated_at": "2021-01-19T20:22:37Z", "author_association": "OWNER", "body": "I can use this test: https://github.com/simonw/datasette/blob/c38c42948cbfddd587729413fd6082ba352eaece/tests/test_plugins.py#L238-L294", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 789336592, "label": "view_name = \"query\" for the query page"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/223#issuecomment-762540514", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/223", "id": 762540514, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjU0MDUxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-19T01:14:58Z", "updated_at": "2021-01-19T01:15:54Z", "author_association": "OWNER", "body": "Example from https://docs.python.org/3/library/csv.html#csv.reader\r\n```pycon\r\n>>> import csv\r\n>>> with open('eggs.csv', newline='') as csvfile:\r\n... spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')\r\n... for row in spamreader:\r\n... print(', '.join(row))\r\nSpam, Spam, Spam, Spam, Spam, Baked Beans\r\nSpam, Lovely Spam, Wonderful Spam\r\n```\r\nI'm going to add `--quotechar` as well as `--delimiter`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 788527932, "label": "--delimiter option for CSV import"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1175#issuecomment-762488336", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1175", "id": 762488336, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjQ4ODMzNg==", "user": {"value": 758858, "label": "hannseman"}, "created_at": "2021-01-18T21:59:28Z", "updated_at": "2021-01-18T22:00:31Z", "author_association": "NONE", "body": "I encountered your issue when trying to find a solution and came up with the following, maybe it can help.\r\n\r\n```python\r\nimport logging.config\r\nfrom typing import Tuple\r\n\r\nimport structlog\r\nimport uvicorn\r\n\r\nfrom example.config import settings\r\n\r\nshared_processors: Tuple[structlog.types.Processor, ...] = (\r\n structlog.contextvars.merge_contextvars,\r\n structlog.stdlib.add_logger_name,\r\n structlog.stdlib.add_log_level,\r\n structlog.processors.TimeStamper(fmt=\"iso\"),\r\n)\r\n\r\nlogging_config = {\r\n \"version\": 1,\r\n \"disable_existing_loggers\": False,\r\n \"formatters\": {\r\n \"json\": {\r\n \"()\": structlog.stdlib.ProcessorFormatter,\r\n \"processor\": structlog.processors.JSONRenderer(),\r\n \"foreign_pre_chain\": shared_processors,\r\n },\r\n \"console\": {\r\n \"()\": structlog.stdlib.ProcessorFormatter,\r\n \"processor\": structlog.dev.ConsoleRenderer(),\r\n \"foreign_pre_chain\": shared_processors,\r\n },\r\n **uvicorn.config.LOGGING_CONFIG[\"formatters\"],\r\n },\r\n \"handlers\": {\r\n \"default\": {\r\n \"level\": \"DEBUG\",\r\n \"class\": \"logging.StreamHandler\",\r\n \"formatter\": \"json\" if not settings.debug else \"console\",\r\n },\r\n \"uvicorn.access\": {\r\n \"level\": \"INFO\",\r\n \"class\": \"logging.StreamHandler\",\r\n \"formatter\": \"access\",\r\n },\r\n \"uvicorn.default\": {\r\n \"level\": \"INFO\",\r\n \"class\": \"logging.StreamHandler\",\r\n \"formatter\": \"default\",\r\n },\r\n },\r\n \"loggers\": {\r\n \"\": {\"handlers\": [\"default\"], \"level\": \"INFO\"},\r\n \"uvicorn.error\": {\r\n \"handlers\": [\"default\" if not settings.debug else \"uvicorn.default\"],\r\n \"level\": \"INFO\",\r\n \"propagate\": False,\r\n },\r\n \"uvicorn.access\": {\r\n \"handlers\": [\"default\" if not settings.debug else \"uvicorn.access\"],\r\n \"level\": \"INFO\",\r\n \"propagate\": False,\r\n },\r\n },\r\n}\r\n\r\n\r\ndef setup_logging() -> None:\r\n structlog.configure(\r\n processors=[\r\n structlog.stdlib.filter_by_level,\r\n *shared_processors,\r\n structlog.stdlib.PositionalArgumentsFormatter(),\r\n structlog.processors.StackInfoRenderer(),\r\n structlog.processors.format_exc_info,\r\n structlog.stdlib.ProcessorFormatter.wrap_for_formatter,\r\n ],\r\n context_class=dict,\r\n logger_factory=structlog.stdlib.LoggerFactory(),\r\n wrapper_class=structlog.stdlib.AsyncBoundLogger,\r\n cache_logger_on_first_use=True,\r\n )\r\n logging.config.dictConfig(logging_config)\r\n```\r\n\r\nAnd then it'll be run on the startup event:\r\n```python\r\n@app.on_event(\"startup\")\r\nasync def startup_event() -> None:\r\n setup_logging()\r\n```\r\n\r\nIt depends on a setting called `debug` which controls if it should output the regular uvicorn logging or json. ", "reactions": "{\"total_count\": 15, \"+1\": 7, \"-1\": 0, \"laugh\": 1, \"hooray\": 1, \"confused\": 0, \"heart\": 5, \"rocket\": 1, \"eyes\": 0}", "issue": {"value": 779156520, "label": "Use structlog for logging"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1036#issuecomment-762391426", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1036", "id": 762391426, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM5MTQyNg==", "user": {"value": 4997607, "label": "philshem"}, "created_at": "2021-01-18T17:45:00Z", "updated_at": "2021-01-18T17:45:00Z", "author_association": "NONE", "body": "It might be possible with this library: https://docs.python.org/3/library/imghdr.html\r\n\r\nquick test of the downloaded blob:\r\n\r\n```\r\n>>> import imghdr\r\n>>> imghdr.what('material_culture-1-image.blob')\r\n'jpeg'\r\n```\r\n\r\nThe output plugin would be cool. I'll look into making my first datasette plugin. I'm also imagining displaying the image in the browser -- but that would be a step 2.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 725996507, "label": "Make it possible to download BLOB data from the Datasette UI"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1194#issuecomment-762390568", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1194", "id": 762390568, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM5MDU2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T17:43:03Z", "updated_at": "2021-01-18T17:43:03Z", "author_association": "OWNER", "body": "Should I just blanket copy over any query string argument that starts with an underscore? Any reason _not_ to do that?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 788447787, "label": "?_size= argument is not persisted by hidden form fields in the table filters"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1194#issuecomment-762390401", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1194", "id": 762390401, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM5MDQwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T17:42:38Z", "updated_at": "2021-01-18T17:42:38Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/a882d679626438ba0d809944f06f239bcba8ee96/datasette/views/table.py#L815-L827\r\n\r\nIt looks like there are other arguments that may not be persisted too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 788447787, "label": "?_size= argument is not persisted by hidden form fields in the table filters"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1036#issuecomment-762387875", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1036", "id": 762387875, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM4Nzg3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T17:36:36Z", "updated_at": "2021-01-18T17:36:36Z", "author_association": "OWNER", "body": "As you can see, I'm pretty paranoid about serving content with `Content-Type` HTTP headers because I'm so worried about execution vulnerabilities. I'm much more comfortable exploring that kind of thing in plugins, since that way people can opt-in to riskier features.\r\n\r\nYou found `datasette-media` which is my most comprehensive exploration of that idea so far - but there's definitely lots of room for more plugins along those lines.\r\n\r\nMaybe even an output plugin? `.jpg` as an export format which returns the `BLOB` column for a row as a JPEG image with the correct `content-type` header (but first verifies that the binary content does indeed look like a real JPEG) could be interesting.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 725996507, "label": "Make it possible to download BLOB data from the Datasette UI"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1036#issuecomment-762385981", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1036", "id": 762385981, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MjM4NTk4MQ==", "user": {"value": 4997607, "label": "philshem"}, "created_at": "2021-01-18T17:32:13Z", "updated_at": "2021-01-18T17:34:50Z", "author_association": "NONE", "body": "Hi Simon\r\n\r\nJust finding this old issue regarding downloading blobs. Nice work!\r\n\r\n\"image\"\r\n\r\nAs a feature request, maybe it would be possible to assign a blob column as a certain data type (e.g. `.jpg`) and then each blob could be downloaded as that type of file (perhaps if the file types were constrained to normal blobs that people store in sqlite databases, this could avoid the execution stuff mentioned above).\r\n\r\nI guess the column blob-type definition could fit into this dropdown selection:\r\n\r\n\"image\"\r\n\r\nLet me know if I should open a new issue with a feature request. (This could slowly go in the direction of displaying image blob-types in the browser.)\r\n\r\nThanks for the great tool!\r\n\r\n\r\n---\r\n\r\nedit: just reading the rest of the twitter thread: https://twitter.com/simonw/status/1318685933256855552\r\n\r\nperhaps this is already possible in some form with the plugin datasette-media: https://github.com/simonw/datasette-media", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 725996507, "label": "Make it possible to download BLOB data from the Datasette UI"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/swarm-to-sqlite/issues/11#issuecomment-761967094", "issue_url": "https://api.github.com/repos/dogsheep/swarm-to-sqlite/issues/11", "id": 761967094, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTk2NzA5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-18T04:11:13Z", "updated_at": "2021-01-18T04:11:13Z", "author_association": "MEMBER", "body": "I just got a similar error:\r\n\r\n```\r\n File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/swarm_to_sqlite/utils.py\", line 79, in save_checkin\r\n checkins_table.m2m(\"users\", user, m2m_table=\"with\", pk=\"id\")\r\n File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 2048, in m2m\r\n id = other_table.insert(record, pk=pk, replace=True).last_pk\r\n File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1781, in insert\r\n return self.insert_all(\r\n File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1899, in insert_all\r\n self.insert_chunk(\r\n File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 1709, in insert_chunk\r\n result = self.db.execute(query, params)\r\n File \"/home/dogsheep/datasette-venv/lib/python3.8/site-packages/sqlite_utils/db.py\", line 226, in execute\r\n return self.conn.execute(sql, parameters)\r\npysqlite3.dbapi2.OperationalError: table users has no column named countryCode\r\n\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 743400216, "label": "Error thrown: sqlite3.OperationalError: table users has no column named lastName"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1191#issuecomment-761705076", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1191", "id": 761705076, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTcwNTA3Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-17T00:35:13Z", "updated_at": "2021-01-17T00:37:51Z", "author_association": "OWNER", "body": "I'm going to try using Jinja macros to implement this: https://jinja.palletsprojects.com/en/2.11.x/templates/#macros\r\n\r\nMaybe using one of these tricks to auto-load the macro? http://codyaray.com/2015/05/auto-load-jinja2-macros", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 787098345, "label": "Ability for plugins to collaborate when adding extra HTML to blocks in default templates"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1181#issuecomment-761703555", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1181", "id": 761703555, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTcwMzU1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-17T00:24:20Z", "updated_at": "2021-01-17T00:24:40Z", "author_association": "OWNER", "body": "Here's the incomplete sketch of a test - to go at the bottom of `test_cli.py`.\r\n\r\n```python\r\n@pytest.mark.parametrize(\r\n \"filename\", [\"test-database (1).sqlite\", \"database (1).sqlite\"]\r\n)\r\ndef test_weird_database_names(ensure_eventloop, tmpdir, filename):\r\n # https://github.com/simonw/datasette/issues/1181\r\n runner = CliRunner()\r\n db_path = str(tmpdir / filename)\r\n sqlite3.connect(db_path).execute(\"vacuum\")\r\n result1 = runner.invoke(cli, [db_path, \"--get\", \"/\"])\r\n assert result1.exit_code == 0, result1.output\r\n homepage_html = result1.output\r\n assert False\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 781262510, "label": "Certain database names results in 404: \"Database not found: None\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1191#issuecomment-761703368", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1191", "id": 761703368, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTcwMzM2OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-17T00:22:46Z", "updated_at": "2021-01-17T00:22:46Z", "author_association": "OWNER", "body": "I'm going to prototype this in a branch.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 787098345, "label": "Ability for plugins to collaborate when adding extra HTML to blocks in default templates"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1191#issuecomment-761703232", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1191", "id": 761703232, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTcwMzIzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-17T00:21:31Z", "updated_at": "2021-01-17T00:21:54Z", "author_association": "OWNER", "body": "I think this ends up being a whole collection of new plugin hooks, something like:\r\n\r\n- `include_table_top`\r\n- `include_table_bottom`\r\n- `include_row_top`\r\n- `include_row_bottom`\r\n- `include_database_top`\r\n- `include_database_bottom`\r\n- `include_query_bottom`\r\n- `include_query_bottom`\r\n- `include_index_bottom`\r\n- `include_index_bottom`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 787098345, "label": "Ability for plugins to collaborate when adding extra HTML to blocks in default templates"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1191#issuecomment-761703022", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1191", "id": 761703022, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTcwMzAyMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-17T00:20:00Z", "updated_at": "2021-01-17T00:20:00Z", "author_association": "OWNER", "body": "Plugins that want to provide extra context to the template can already do so using the `extra_template_vars()` plugin hook.\r\n\r\nSo this hook could work by returning a list of template filenames to be included. Those templates can be bundled with the plugin. Since they are included they will have access to the template context and to any `extra_template_vars()` values.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 787098345, "label": "Ability for plugins to collaborate when adding extra HTML to blocks in default templates"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/657#issuecomment-761179229", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/657", "id": 761179229, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTE3OTIyOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-15T20:24:35Z", "updated_at": "2021-01-15T20:24:35Z", "author_association": "OWNER", "body": "I'm not sure how I missed this issue but it's almost a year later and I'm finally taking a look at your Parquet work.\r\n\r\nThis is yet more evidence that allowing plugins to provide their own custom `Database` objects would be a good idea.\r\n\r\nI started exploring what Datasette would like on PostgreSQL in #670 - my concern was that I would need to add a large amount of database abstraction code which would dramatically increase the complexity of the core project, but my thinking now is that it might be tractable - Datasette doesn't actually construct SQL in complex ways anywhere outside of the `TableView` class so abstracting away just that bit should be feasible.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 548591089, "label": "Allow creation of virtual tables at startup"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1191#issuecomment-761103910", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1191", "id": 761103910, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTEwMzkxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-15T18:19:29Z", "updated_at": "2021-01-15T18:19:29Z", "author_association": "OWNER", "body": "This relates to #987 (documented HTML hooks for JavaScript plugins) but is not quite the same thing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 787098345, "label": "Ability for plugins to collaborate when adding extra HTML to blocks in default templates"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/657#issuecomment-761101878", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/657", "id": 761101878, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTEwMTg3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-15T18:16:01Z", "updated_at": "2021-01-15T18:16:01Z", "author_association": "OWNER", "body": "I think the `startup()` plugin hook at https://docs.datasette.io/en/stable/plugin_hooks.html#startup-datasette should be able to fit this. You can write a plugin which uses that hook to execute `CREATE VIRTUAL TABLE` against one or more databases when Datasette first starts running.\r\n\r\nWould that work here?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 548591089, "label": "Allow creation of virtual tables at startup"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/220#issuecomment-761015218", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/220", "id": 761015218, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MTAxNTIxOA==", "user": {"value": 649467, "label": "mhalle"}, "created_at": "2021-01-15T15:40:08Z", "updated_at": "2021-01-15T15:40:08Z", "author_association": "NONE", "body": "Make sense. If you're coming from the sqlite3 side of things, rather than the datasette side, wanting the fts methods to work for views makes more sense. sqlite3 allows fts5 tables on views, so I was looking for CLI functionality to build the fts virtual tables. Ultimately, though, sharing fts virtual tables across tables and derivative views is likely more efficient. \r\n\r\nMaybe an explicit error message like, \"fts is not supported for views\" rather than just throwing an exception that the method doesn't exist\" might be helpful. Not critical though.\r\n\r\nThanks.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 783778672, "label": "Better error message for *_fts methods against views"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/twitter-to-sqlite/pull/55#issuecomment-760950128", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/55", "id": 760950128, "node_id": "MDEyOklzc3VlQ29tbWVudDc2MDk1MDEyOA==", "user": {"value": 21148, "label": "jacobian"}, "created_at": "2021-01-15T13:44:52Z", "updated_at": "2021-01-15T13:44:52Z", "author_association": "CONTRIBUTOR", "body": "I found and fixed another bug, this one around importing the tweets table. @simonw let me know if you'd prefer this broken out into multiple PRs, happy to do that if it makes review/merging easier.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779211940, "label": "Fix archive imports"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1187#issuecomment-759875239", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1187", "id": 759875239, "node_id": "MDEyOklzc3VlQ29tbWVudDc1OTg3NTIzOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-14T02:02:24Z", "updated_at": "2021-01-14T02:02:31Z", "author_association": "OWNER", "body": "This plugin hook currently returns a string of JavaScript. It could optionally return this instead of a string:\r\n```json\r\n{\r\n \"script\": \"string of JavaScript goes here\",\r\n \"module\": true\r\n}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 785588942, "label": "extra_body_script() support for script type=\"module\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1186#issuecomment-759874332", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1186", "id": 759874332, "node_id": "MDEyOklzc3VlQ29tbWVudDc1OTg3NDMzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-14T01:59:35Z", "updated_at": "2021-01-14T01:59:35Z", "author_association": "OWNER", "body": "Updated documentation: https://docs.datasette.io/en/latest/custom_templates.html#custom-css-and-javascript and https://docs.datasette.io/en/latest/plugin_hooks.html#extra-js-urls-template-database-table-columns-view-name-request-datasette", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 785573793, "label": "script type=\"module\" support"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/1159#issuecomment-759306228", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1159", "id": 759306228, "node_id": "MDEyOklzc3VlQ29tbWVudDc1OTMwNjIyOA==", "user": {"value": 552629, "label": "lovasoa"}, "created_at": "2021-01-13T08:59:31Z", "updated_at": "2021-01-13T08:59:31Z", "author_association": "NONE", "body": "@simonw : Did you have the time to take a look at this ?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 774332247, "label": "Improve the display of facets information"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/220#issuecomment-759098964", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/220", "id": 759098964, "node_id": "MDEyOklzc3VlQ29tbWVudDc1OTA5ODk2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-12T23:19:55Z", "updated_at": "2021-01-12T23:19:55Z", "author_association": "OWNER", "body": "I don't think it makes sense to call `.enable_fts()` on a view does it? When I'm working with views and FTS I tend to write my queries against a FTS table for one of the tables that is used by the view - https://docs.datasette.io/en/stable/full_text_search.html#configuring-full-text-search-for-a-table-or-view describes how I do that in Datasette for example.\r\n\r\nCan you expand on your use-case for FTS and views? I'm ready to be convinced otherwise, but I don't see how it would work right now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 783778672, "label": "Better error message for *_fts methods against views"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1185#issuecomment-759069342", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1185", "id": 759069342, "node_id": "MDEyOklzc3VlQ29tbWVudDc1OTA2OTM0Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-12T22:13:18Z", "updated_at": "2021-01-12T22:13:18Z", "author_association": "OWNER", "body": "I'm going to change the error message to list the allowed pragmas.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 784628163, "label": "\"Statement may not contain PRAGMA\" error is not strictly true"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1185#issuecomment-759067427", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1185", "id": 759067427, "node_id": "MDEyOklzc3VlQ29tbWVudDc1OTA2NzQyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-12T22:09:21Z", "updated_at": "2021-01-12T22:09:21Z", "author_association": "OWNER", "body": "That allow-list was added in #761 but is not currently documented. It's here in the code:\r\n\r\nhttps://github.com/simonw/datasette/blob/8e8fc5cee5c78da8334495c6d6257d5612c40792/datasette/utils/__init__.py#L173-L186", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 784628163, "label": "\"Statement may not contain PRAGMA\" error is not strictly true"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1185#issuecomment-759066777", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1185", "id": 759066777, "node_id": "MDEyOklzc3VlQ29tbWVudDc1OTA2Njc3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-12T22:07:58Z", "updated_at": "2021-01-12T22:07:58Z", "author_association": "OWNER", "body": "https://docs.datasette.io/en/stable/sql_queries.html?highlight=pragma#named-parameters documentation is out-of-date as well:\r\n\r\n> Datasette disallows custom SQL containing the string PRAGMA, as SQLite pragma statements can be used to change database settings at runtime. If you need to include the string \"pragma\" in a query you can do so safely using a named parameter.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 784628163, "label": "\"Statement may not contain PRAGMA\" error is not strictly true"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1091#issuecomment-758668359", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1091", "id": 758668359, "node_id": "MDEyOklzc3VlQ29tbWVudDc1ODY2ODM1OQ==", "user": {"value": 6739646, "label": "tballison"}, "created_at": "2021-01-12T13:52:29Z", "updated_at": "2021-01-12T13:52:29Z", "author_association": "NONE", "body": "Y, thank you to both of you!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 742011049, "label": ".json and .csv exports fail to apply base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1091#issuecomment-758448525", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1091", "id": 758448525, "node_id": "MDEyOklzc3VlQ29tbWVudDc1ODQ0ODUyNQ==", "user": {"value": 19328961, "label": "henry501"}, "created_at": "2021-01-12T06:55:08Z", "updated_at": "2021-01-12T06:55:08Z", "author_association": "NONE", "body": "Great, really happy I could help! Reverse proxies get tricky.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 742011049, "label": ".json and .csv exports fail to apply base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1183#issuecomment-758356640", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1183", "id": 758356640, "node_id": "MDEyOklzc3VlQ29tbWVudDc1ODM1NjY0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-12T02:42:08Z", "updated_at": "2021-01-12T02:42:08Z", "author_association": "OWNER", "body": "Should Datasette have subcommands for this? `datasette enable-counts data.db` and `datasette disable-counts data.db` and `datasette reset-counts data.db` commands?\r\n\r\nMaybe. The `sqlite-utils` CLI tool could be used here instead, but that won't be easily available if Datasette was installed as a standalone binary or using `brew install datasette` or `pipx install datasette`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 782708469, "label": "Take advantage of sqlite-utils cached table counts, if available"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1183#issuecomment-758356097", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1183", "id": 758356097, "node_id": "MDEyOklzc3VlQ29tbWVudDc1ODM1NjA5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-12T02:40:30Z", "updated_at": "2021-01-12T02:40:30Z", "author_association": "OWNER", "body": "So how would this work?\r\n\r\nI think I'm going to automatically use these values if the `_counts` table exists, unless a `ignore_counts_table` boolean setting has been set. I won't bother looking to see if the triggers have been created.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 782708469, "label": "Take advantage of sqlite-utils cached table counts, if available"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1091#issuecomment-758283074", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1091", "id": 758283074, "node_id": "MDEyOklzc3VlQ29tbWVudDc1ODI4MzA3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-11T23:12:46Z", "updated_at": "2021-01-11T23:12:46Z", "author_association": "OWNER", "body": "Fantastic!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 742011049, "label": ".json and .csv exports fail to apply base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1091#issuecomment-758280611", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1091", "id": 758280611, "node_id": "MDEyOklzc3VlQ29tbWVudDc1ODI4MDYxMQ==", "user": {"value": 6739646, "label": "tballison"}, "created_at": "2021-01-11T23:06:10Z", "updated_at": "2021-01-11T23:06:10Z", "author_association": "NONE", "body": "+1\r\n\r\nYep! Fixes it. If I navigate to https://corpora.tika.apache.org/datasette, I get a 404 (database not found: datasette), but if I navigate to https://corpora.tika.apache.org/datasette/file_profiles/, everything WORKS!\r\n\r\nThank you!", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 1, \"eyes\": 0}", "issue": {"value": 742011049, "label": ".json and .csv exports fail to apply base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1182#issuecomment-757375858", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1182", "id": 757375858, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NzM3NTg1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-09T22:18:47Z", "updated_at": "2021-01-09T22:18:47Z", "author_association": "OWNER", "body": "https://docs.datasette.io/en/latest/ecosystem.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 782692159, "label": "Retire \"Ecosystem\" page in favour of datasette.io/plugins and /tools"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1182#issuecomment-757373741", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1182", "id": 757373741, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NzM3Mzc0MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-09T22:01:41Z", "updated_at": "2021-01-09T22:01:41Z", "author_association": "OWNER", "body": "It can talk about Dogsheep too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 782692159, "label": "Retire \"Ecosystem\" page in favour of datasette.io/plugins and /tools"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1182#issuecomment-757373082", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1182", "id": 757373082, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NzM3MzA4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-09T21:55:33Z", "updated_at": "2021-01-09T21:55:33Z", "author_association": "OWNER", "body": "I'll leave the page there but change it into more of a blurb about the existence of the plugins and tools directories.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 782692159, "label": "Retire \"Ecosystem\" page in favour of datasette.io/plugins and /tools"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1181#issuecomment-756487966", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1181", "id": 756487966, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NjQ4Nzk2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-08T01:25:42Z", "updated_at": "2021-01-08T01:25:42Z", "author_association": "OWNER", "body": "I'm going to add a unit test that tries a variety of weird database names.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 781262510, "label": "Certain database names results in 404: \"Database not found: None\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1181#issuecomment-756482163", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1181", "id": 756482163, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NjQ4MjE2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-08T01:06:23Z", "updated_at": "2021-01-08T01:06:54Z", "author_association": "OWNER", "body": "Yes, that logic is definitely at fault. It looks like it applies `urllib.parse.unquote_plus()` AFTER it's tried to do the `-` hash splitting thing, which is why it's failing here:\r\n\r\nhttps://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/views/base.py#L184-L198", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 781262510, "label": "Certain database names results in 404: \"Database not found: None\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1091#issuecomment-756453945", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1091", "id": 756453945, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NjQ1Mzk0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-07T23:42:50Z", "updated_at": "2021-01-07T23:42:50Z", "author_association": "OWNER", "body": "@henry501 it looks like you spotted a bug in the documentation - I just addressed that, the fix is now live here: https://docs.datasette.io/en/latest/deploying.html#running-datasette-behind-a-proxy", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 1, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 742011049, "label": ".json and .csv exports fail to apply base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1091#issuecomment-756453010", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1091", "id": 756453010, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NjQ1MzAxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-07T23:39:58Z", "updated_at": "2021-01-07T23:40:25Z", "author_association": "OWNER", "body": "@tballison I think that's the solution! It looks like you need to use this in your config:\r\n\r\n`ProxyPass /datasette http://127.0.0.1:8001/datasette`\r\n\r\nInstead of this:\r\n\r\n`ProxyPass /datasette http://127.0.0.1:8001/`\r\n\r\nGive that a go and let me know if it fixes it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 742011049, "label": ".json and .csv exports fail to apply base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1091#issuecomment-756425587", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1091", "id": 756425587, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NjQyNTU4Nw==", "user": {"value": 19328961, "label": "henry501"}, "created_at": "2021-01-07T22:27:19Z", "updated_at": "2021-01-07T22:27:19Z", "author_association": "NONE", "body": "I found this issue while troubleshooting the same behavior with an nginx reverse proxy. The solution was to make sure I set:\r\n\r\n`proxy_pass http://server:8001/baseurl/ \r\n`\r\ninstead of just:\r\n\r\n`proxy_pass http://server:8001\r\n`\r\nThe custom SQL query and header links are now correct.\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 742011049, "label": ".json and .csv exports fail to apply base_url"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-756335394", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 756335394, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NjMzNTM5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-07T19:35:59Z", "updated_at": "2021-01-07T19:35:59Z", "author_association": "OWNER", "body": "I requested a D-U-N-S number as a first step in getting a developer certificate: https://developer.apple.com/support/D-U-N-S/\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1180#issuecomment-756312213", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1180", "id": 756312213, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NjMxMjIxMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-07T18:56:24Z", "updated_at": "2021-01-07T18:56:24Z", "author_association": "OWNER", "body": "The `async_call_with_supported_arguments` version should be able to await any of the lazy arguments that are awaitable - can use `await_me_maybe` for that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780767542, "label": "Lazily evaluated arguments for call_with_supported_arguments"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1180#issuecomment-755500475", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1180", "id": 755500475, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTUwMDQ3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:43:41Z", "updated_at": "2021-01-06T18:43:41Z", "author_association": "OWNER", "body": "Relevant code: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/utils/__init__.py#L919-L940", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780767542, "label": "Lazily evaluated arguments for call_with_supported_arguments"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1179#issuecomment-755495387", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1179", "id": 755495387, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTQ5NTM4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:39:23Z", "updated_at": "2021-01-06T18:39:23Z", "author_association": "OWNER", "body": "In that case maybe there are three new arguments: `path`, `full_path` and `url`.\r\n\r\nI'll also add `request.full_path` for consistency with these: https://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/utils/asgi.py#L77-L90", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780278550, "label": "Make original path available to render hooks"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1179#issuecomment-755492945", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1179", "id": 755492945, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTQ5Mjk0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:37:39Z", "updated_at": "2021-01-06T18:37:39Z", "author_association": "OWNER", "body": "I think I'll call this `full_path` for consistency with Django.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780278550, "label": "Make original path available to render hooks"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1179#issuecomment-755489974", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1179", "id": 755489974, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTQ4OTk3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:35:24Z", "updated_at": "2021-01-06T18:35:24Z", "author_association": "OWNER", "body": "Django calls this ` HttpRequest.get_full_path()` - for the path plus the querystring.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780278550, "label": "Make original path available to render hooks"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1179#issuecomment-755486103", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1179", "id": 755486103, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTQ4NjEwMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:32:41Z", "updated_at": "2021-01-06T18:34:11Z", "author_association": "OWNER", "body": "This parameter will return the URL path, with querystring arguments, to the HTML version of the page - e.g. `/github/issue_comments` or `/github/issue_comments?_sort_desc=created_at`\r\n\r\nOpen questions:\r\n\r\n- What should it be called? `path` could be misleading since it also includes the querystring.\r\n- Should I provide a `url` or `full_url` version which includes `https://blah.com/...`?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780278550, "label": "Make original path available to render hooks"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-755484384", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 755484384, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTQ4NDM4NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:31:14Z", "updated_at": "2021-01-06T18:31:57Z", "author_association": "OWNER", "body": "In building https://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_labels=on I discovered the following patterns for importing data into both Pandas and Observable/d3:\r\n```python\r\nimport pandas\r\ndf = pandas.read_json(\r\n \"https://latest-with-plugins.datasette.io/github/issue_comments.json?_shape=array\"\r\n)\r\n```\r\nAnd:\r\n```javascript\r\nd3 = require(\"d3@5\")\r\nrows = d3.json(\r\n \"https://latest-with-plugins.datasette.io/github/issue_comments.json?_shape=array\"\r\n)\r\n```\r\nOnce again I find myself torn on the best possible default. A list of JSON objects is instantly compatible with both `pandas.read_json()` and `d3.json()` - but it leaves nowhere to put the extra information like pagination and suchlike!\r\n\r\nEven given this I still think the correct default is an object with `\"rows\"`, `\"total\"` and `\"next_url\"` keys. I should commit to that and implement it - this thought exercise has been running for far too long.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755476820", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755476820, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTQ3NjgyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:24:47Z", "updated_at": "2021-01-06T18:24:47Z", "author_association": "OWNER", "body": "Issue fixed - https://latest-with-plugins.datasette.io/github/issue_comments.Notebook?_labels=on displays the correct schemes now.\r\n\r\nI can't think of a reason anyone on Cloud Run would ever NOT want the `force_https_urls` option, but just in case I've made it so if you pass `--extra-options --setting force_https_urls off` to `publish cloudrun` your setting will be respected.\r\n\r\nhttps://github.com/simonw/datasette/blob/97fb10c17dd007a275ab743742e93e932335ad67/datasette/publish/cloudrun.py#L105-L110", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755468795", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755468795, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTQ2ODc5NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T18:14:35Z", "updated_at": "2021-01-06T18:14:35Z", "author_association": "OWNER", "body": "Deploying that change now to test it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755163886", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755163886, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE2Mzg4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:37:51Z", "updated_at": "2021-01-06T08:37:51Z", "author_association": "OWNER", "body": "Easiest fix would be for `publish cloudrun` to set `force_https_urls`:\r\n\r\n`datasette publish now` used to do this: https://github.com/simonw/datasette/blob/07e208cc6d9e901b87552c1be2854c220b3f9b6d/datasette/publish/now.py#L59-L63", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1179#issuecomment-755161574", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1179", "id": 755161574, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE2MTU3NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:32:31Z", "updated_at": "2021-01-06T08:32:31Z", "author_association": "OWNER", "body": "An optional `path` argument to https://docs.datasette.io/en/stable/plugin_hooks.html#register-output-renderer-datasette which shows the path WITHOUT the `.Notebook` extension would be useful here.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780278550, "label": "Make original path available to render hooks"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755160187", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755160187, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE2MDE4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:29:35Z", "updated_at": "2021-01-06T08:29:35Z", "author_association": "OWNER", "body": "https://latest-with-plugins.datasette.io/-/asgi-scope\r\n\r\n```\r\n{'asgi': {'spec_version': '2.1', 'version': '3.0'},\r\n 'client': ('169.254.8.129', 54971),\r\n 'headers': [(b'host', b'latest-with-plugins.datasette.io'),\r\n (b'user-agent',\r\n b'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:84.0) Gecko'\r\n b'/20100101 Firefox/84.0'),\r\n (b'accept',\r\n b'text/html,application/xhtml+xml,application/xml;q=0.9,image/'\r\n b'webp,*/*;q=0.8'),\r\n (b'accept-language', b'en-US,en;q=0.5'),\r\n (b'dnt', b'1'),\r\n (b'cookie',\r\n b'_ga_LL6M7BK6D4=GS1.1.1609886546.49.1.1609886923.0; _ga=GA1.1'\r\n b'.894633707.1607575712'),\r\n (b'upgrade-insecure-requests', b'1'),\r\n (b'x-client-data', b'CgSL6ZsV'),\r\n (b'x-cloud-trace-context',\r\n b'e776af843c657d2a3da28a73b726e6fe/14187666787557102189;o=1'),\r\n (b'x-forwarded-for', b'148.64.98.14'),\r\n (b'x-forwarded-proto', b'https'),\r\n (b'forwarded', b'for=\"148.64.98.14\";proto=https'),\r\n (b'accept-encoding', b'gzip, deflate, br'),\r\n (b'content-length', b'0')],\r\n 'http_version': '1.1',\r\n 'method': 'GET',\r\n 'path': '/-/asgi-scope',\r\n 'query_string': b'',\r\n 'raw_path': b'/-/asgi-scope',\r\n 'root_path': '',\r\n 'scheme': 'http',\r\n 'server': ('169.254.8.130', 8080),\r\n 'type': 'http'}\r\n```\r\nNote the `'scheme': 'http'` but also the `(b'x-forwarded-proto', b'https')`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1176#issuecomment-755159583", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1176", "id": 755159583, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE1OTU4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:28:20Z", "updated_at": "2021-01-06T08:28:20Z", "author_association": "OWNER", "body": "I used `from datasette.utils import path_with_format` in https://github.com/simonw/datasette-export-notebook/blob/0.1/datasette_export_notebook/__init__.py just now.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779691739, "label": "Policy on documenting \"public\" datasette.utils functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755158310", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755158310, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE1ODMxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:25:31Z", "updated_at": "2021-01-06T08:25:31Z", "author_association": "OWNER", "body": "Moving this to the Datasette repo.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755157732", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755157732, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE1NzczMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:24:12Z", "updated_at": "2021-01-06T08:24:12Z", "author_association": "OWNER", "body": "https://latest-with-plugins.datasette.io/fixtures/sortable.json has the bug too - the `next_url` is `http://` when it should be `https://`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755157281", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755157281, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE1NzI4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:23:14Z", "updated_at": "2021-01-06T08:23:14Z", "author_association": "OWNER", "body": "https://latest-with-plugins.datasette.io/-/settings says `\"force_https_urls\": false`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755157066", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755157066, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE1NzA2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:22:47Z", "updated_at": "2021-01-06T08:22:47Z", "author_association": "OWNER", "body": "Weird... https://github.com/simonw/datasette/blob/a882d679626438ba0d809944f06f239bcba8ee96/datasette/app.py#L609-L613\r\n\r\n```python\r\n def absolute_url(self, request, path):\r\n url = urllib.parse.urljoin(request.url, path)\r\n if url.startswith(\"http://\") and self.setting(\"force_https_urls\"):\r\n url = \"https://\" + url[len(\"http://\") :]\r\n return url\r\n```\r\nThat looks like it should work. Needs more digging.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1178#issuecomment-755156606", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1178", "id": 755156606, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTE1NjYwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T08:21:49Z", "updated_at": "2021-01-06T08:21:49Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette-export-notebook/blob/aec398eab4f34791d240d7bc47b6eec575b357be/datasette_export_notebook/__init__.py#L18-L23\r\n\r\nMaybe this is a bug in `datasette.absolute_url`? Perhaps it doesn't take the scheme into account.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 780267857, "label": "Use force_https_urls on when deploying with Cloud Run"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1101#issuecomment-755134771", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1101", "id": 755134771, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTEzNDc3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T07:28:01Z", "updated_at": "2021-01-06T07:28:01Z", "author_association": "OWNER", "body": "With this structure it will become possible to stream non-newline-delimited JSON array-of-objects too - the `stream_rows()` method could output `[` first, then each row followed by a comma, then `]` after the very last row.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 749283032, "label": "register_output_renderer() should support streaming data"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1101#issuecomment-755133937", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1101", "id": 755133937, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTEzMzkzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T07:25:48Z", "updated_at": "2021-01-06T07:26:43Z", "author_association": "OWNER", "body": "Idea: instead of returning a dictionary, `register_output_renderer` could return an object. The object could have the following properties:\r\n\r\n- `.extension` - the extension to use\r\n- `.can_render(...)` - says if it can render this\r\n- `.can_stream(...)` - says if streaming is supported\r\n- `async .stream_rows(rows_iterator, send)` - method that loops through all rows and uses `send` to send them to the response in the correct format\r\n\r\nI can then deprecate the existing `dict` return type for 1.0.", "reactions": "{\"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 749283032, "label": "register_output_renderer() should support streaming data"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1101#issuecomment-755128038", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1101", "id": 755128038, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NTEyODAzOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-06T07:10:22Z", "updated_at": "2021-01-06T07:10:22Z", "author_association": "OWNER", "body": "Yet another use-case for this: I want to be able to stream newline-delimited JSON in order to better import into Pandas:\r\n\r\n pandas.read_json(\"https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_nl=on\", lines=True)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 749283032, "label": "register_output_renderer() should support streaming data"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-754958998", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 754958998, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1ODk5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T23:16:33Z", "updated_at": "2021-01-05T23:16:33Z", "author_association": "OWNER", "body": "That's really useful, thanks @rcoup ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/782#issuecomment-754958610", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/782", "id": 754958610, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1ODYxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T23:15:24Z", "updated_at": "2021-01-05T23:15:24Z", "author_association": "OWNER", "body": "https://latest-with-plugins.datasette.io/fixtures/roadside_attraction_characteristics/1.json-preview returns a 500 error at the moment - a KeyError on 'filtered_table_rows_count'.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 627794879, "label": "Redesign default .json format"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/576#issuecomment-754957658", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/576", "id": 754957658, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1NzY1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T23:12:50Z", "updated_at": "2021-01-05T23:12:50Z", "author_association": "OWNER", "body": "See https://docs.datasette.io/en/stable/internals.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 497170355, "label": "Documented internals API for use in plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/576#issuecomment-754957563", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/576", "id": 754957563, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1NzU2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T23:12:37Z", "updated_at": "2021-01-05T23:12:37Z", "author_association": "OWNER", "body": "I'm happy with how this has evolved, so I'm closing the issue.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 497170355, "label": "Documented internals API for use in plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1176#issuecomment-754957378", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1176", "id": 754957378, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1NzM3OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T23:12:03Z", "updated_at": "2021-01-05T23:12:03Z", "author_association": "OWNER", "body": "This needs to be done for Datasette 1.0. At the very least I need to ensure it's clear that `datasette.utils` is not part of the public API unless explicitly marked as such.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779691739, "label": "Policy on documenting \"public\" datasette.utils functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1176#issuecomment-754952146", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1176", "id": 754952146, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1MjE0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T22:57:26Z", "updated_at": "2021-01-05T22:57:26Z", "author_association": "OWNER", "body": "Known public APIs might be worth adding type annotations to as well.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779691739, "label": "Policy on documenting \"public\" datasette.utils functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1176#issuecomment-754952040", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1176", "id": 754952040, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1MjA0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T22:57:09Z", "updated_at": "2021-01-05T22:57:09Z", "author_association": "OWNER", "body": "It might be neater to move all of the non-public functions into a separate module - `datasette.utils.internal` perhaps.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779691739, "label": "Policy on documenting \"public\" datasette.utils functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1176#issuecomment-754951786", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1176", "id": 754951786, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDk1MTc4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T22:56:27Z", "updated_at": "2021-01-05T22:56:43Z", "author_association": "OWNER", "body": "Idea: introduce a `@documented` decorator which marks specific functions as part of the public, documented API. The unit tests can then confirm that anything with that decorator is both documented and tested.\r\n```python\r\n@documented\r\ndef escape_css_string(s):\r\n return _css_re.sub(\r\n lambda m: \"\\\\\" + (f\"{ord(m.group()):X}\".zfill(6)),\r\n s.replace(\"\\r\\n\", \"\\n\"),\r\n )\r\n```\r\nOr maybe `@public`?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779691739, "label": "Policy on documenting \"public\" datasette.utils functions"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-754911290", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 754911290, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDkxMTI5MA==", "user": {"value": 59874, "label": "rcoup"}, "created_at": "2021-01-05T21:31:15Z", "updated_at": "2021-01-05T21:31:15Z", "author_association": "NONE", "body": "We did this for [Sno](https://sno.earth) under macOS \u2014 it's a PyInstaller binary/setup which uses [Packages](http://s.sudre.free.fr/Software/Packages/about.html) for packaging.\r\n\r\n* [Building & Signing](https://github.com/koordinates/sno/blob/master/platforms/Makefile#L67-L95)\r\n* [Packaging & Notarizing](https://github.com/koordinates/sno/blob/master/platforms/Makefile#L121-L215)\r\n* [Github Workflow](https://github.com/koordinates/sno/blob/master/.github/workflows/build.yml#L228-L269) has the CI side of it\r\n\r\nFYI (if you ever get to it) for Windows you need to get a code signing certificate. And if you want automated CI, you'll want to get an \"EV CodeSigning for HSM\" certificate from GlobalSign, which then lets you put the certificate into Azure Key Vault. Which you can use with [azuresigntool](https://github.com/vcsjones/AzureSignTool) to sign your code & installer. (Non-EV certificates are a waste of time, the user still gets big warnings at install time).\r\n", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-754729035", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54", "id": 754729035, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDcyOTAzNQ==", "user": {"value": 21148, "label": "jacobian"}, "created_at": "2021-01-05T16:03:29Z", "updated_at": "2021-01-05T16:03:29Z", "author_association": "CONTRIBUTOR", "body": "I was able to fix this, at least enough to get _my_ archive to import. Not sure if there's more work to be done here or not.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779088071, "label": "Archive import appears to be broken on recent exports"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/twitter-to-sqlite/pull/55#issuecomment-754728696", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/55", "id": 754728696, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDcyODY5Ng==", "user": {"value": 21148, "label": "jacobian"}, "created_at": "2021-01-05T16:02:55Z", "updated_at": "2021-01-05T16:02:55Z", "author_association": "CONTRIBUTOR", "body": "This now works for me, though I'm entirely ensure if it's a just-my-export thing or a wider issue. Also, this doesn't contain any tests. So I'm not sure if there's more work to be done here, or if this is good enough.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779211940, "label": "Fix archive imports"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/twitter-to-sqlite/issues/54#issuecomment-754721153", "issue_url": "https://api.github.com/repos/dogsheep/twitter-to-sqlite/issues/54", "id": 754721153, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDcyMTE1Mw==", "user": {"value": 21148, "label": "jacobian"}, "created_at": "2021-01-05T15:51:09Z", "updated_at": "2021-01-05T15:51:09Z", "author_association": "CONTRIBUTOR", "body": "Correction: the failure is on `lists-member.js` (I was thrown by the `block` variable name, but that's just a coincidence)", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779088071, "label": "Archive import appears to be broken on recent exports"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1175#issuecomment-754696725", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1175", "id": 754696725, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDY5NjcyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T15:12:30Z", "updated_at": "2021-01-05T15:12:30Z", "author_association": "OWNER", "body": "Some tips here: https://github.com/tiangolo/fastapi/issues/78", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 779156520, "label": "Use structlog for logging"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1167#issuecomment-754619930", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1167", "id": 754619930, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDYxOTkzMA==", "user": {"value": 3637, "label": "benpickles"}, "created_at": "2021-01-05T12:57:57Z", "updated_at": "2021-01-05T12:57:57Z", "author_association": "CONTRIBUTOR", "body": "Not sure where exactly to put the actual docs (presumably somewhere in [docs/contributing.rst](https://github.com/simonw/datasette/blob/main/docs/contributing.rst)) but I've made a slight change to make it easier to run locally (copying [the approach in excalidraw](https://github.com/excalidraw/excalidraw/blob/ade2565f497243a5e428f4906d8ed80c872fd981/package.json#L90-L94)): https://github.com/simonw/datasette/compare/main...benpickles:prettier-docs\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 777145954, "label": "Add Prettier to contributing documentation"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/859#issuecomment-647922203", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/859", "id": 647922203, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NzkyMjIwMw==", "user": {"value": 3243482, "label": "abdusco"}, "created_at": "2020-06-23T05:44:58Z", "updated_at": "2021-01-05T08:22:43Z", "author_association": "CONTRIBUTOR", "body": "I'm seeing the problem on database page. Index page and table page runs quite fast.\r\n\r\n- Tables have <10 columns (`id`, `url`, `title`, `body_html`, `date`, `author`, `meta` (for keeping unstructured json)). I've added index on `date` columns (using `sqlite-utils`) in addition to the index present on `id` columns. \r\n- All tables have FTS enabled on `text` and `varchar` columns (`title`, `body_html` etc) to speed up searching.\r\n- There are couple of tables related with foreign keys (think a thread in a forum and posts in that thread, related with `thread_id`)\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 642572841, "label": "Database page loads too slowly with many large tables (due to table counts)"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1173#issuecomment-754463845", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1173", "id": 754463845, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDQ2Mzg0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-05T07:41:43Z", "updated_at": "2021-01-05T07:41:43Z", "author_association": "OWNER", "body": "https://github.com/oleksis/pyinstaller-manylinux looks useful, via https://twitter.com/oleksis/status/1346341987876823040", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778682317, "label": "GitHub Actions workflow to build manylinux binary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-754296761", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 754296761, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDI5Njc2MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T23:55:44Z", "updated_at": "2021-01-04T23:55:44Z", "author_association": "OWNER", "body": "Bit uncomfortable that it looks like you need to include your Apple ID username and password in the CI configuration to do this. I'll use GitHub Secrets for this but I don't like it - I'll definitely setup a dedicated code signing account that's not my access-to-everything AppleID for this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-754295380", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 754295380, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDI5NTM4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T23:54:32Z", "updated_at": "2021-01-04T23:54:32Z", "author_association": "OWNER", "body": "https://github.com/search?l=YAML&q=gon+json&type=Code reveals some examples of people using `gon` in workflows.\r\n\r\nThese look useful:\r\n\r\n* https://github.com/coherence/hub-server/blob/3b7e9c7c5bce9e244b14b854f1f89d66f53a5a39/.github/workflows/release_build.yml\r\n* https://github.com/simoncozens/pilcrow/blob/5abc145e7fb9577086afe47b48fd730cb8195386/.github/workflows/buildapp.yaml", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-754287882", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 754287882, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDI4Nzg4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T23:40:10Z", "updated_at": "2021-01-04T23:42:32Z", "author_association": "OWNER", "body": "This looks VERY useful: https://github.com/mitchellh/gon - \" Sign, notarize, and package macOS CLI tools and applications written in any language. Available as both a CLI and a Go library.\"\r\n\r\nAnd it installs like this:\r\n\r\n brew install mitchellh/gon/gon", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-754286783", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 754286783, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDI4Njc4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T23:38:18Z", "updated_at": "2021-01-04T23:38:18Z", "author_association": "OWNER", "body": "Oh wow maybe I need to Notarize it too? https://developer.apple.com/documentation/xcode/notarizing_macos_software_before_distribution", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1171#issuecomment-754286618", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1171", "id": 754286618, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDI4NjYxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T23:37:45Z", "updated_at": "2021-01-04T23:37:45Z", "author_association": "OWNER", "body": "https://github.com/actions/virtual-environments/issues/1820#issuecomment-719549887 looks useful - not sure if those notes are for iOS or macOS though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 778450486, "label": "GitHub Actions workflow to build and sign macOS binary executables"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/93#issuecomment-754285795", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/93", "id": 754285795, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDI4NTc5NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T23:35:13Z", "updated_at": "2021-01-04T23:35:13Z", "author_association": "OWNER", "body": "Next step is to automate this all!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 273944952, "label": "Package as standalone binary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/1152#issuecomment-754285588", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1152", "id": 754285588, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDI4NTU4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T23:34:30Z", "updated_at": "2021-01-04T23:34:30Z", "author_association": "OWNER", "body": "I think the way to do this is to have a new plugin hook that returns two SQL where clauses: one returning a list of resources that the user should be able to access (the allow-list) and one returning a list of resources they are explicitly forbidden from accessing (the deny-list). Either of these can be blank.\r\n\r\nDatasette can then combine those into a full SQL query and use it to answer the question \"show me a list of resources that the user is allowed to perform action X on\". It can also answer the existing question, \"is user X allowed to perform action Y on resource Z\"?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 770598024, "label": "Efficiently calculate list of databases/tables a user can view"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/93#issuecomment-754233960", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/93", "id": 754233960, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDIzMzk2MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T21:35:37Z", "updated_at": "2021-01-04T21:35:37Z", "author_association": "OWNER", "body": "I tested it by running a `tmate` session on the GitHub macOS machines, and it worked!\r\n```\r\nMac-1609795972770:tmp runner$ wget 'https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip'\r\n--2021-01-04 21:34:10-- https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip\r\nResolving github.com (github.com)... 140.82.114.4\r\nConnecting to github.com (github.com)|140.82.114.4|:443... connected.\r\nHTTP request sent, awaiting response... 302 Found\r\nLocation: https://github-production-release-asset-2e65be.s3.amazonaws.com/107914493/74658700-4e90-11eb-8f3b-ee77e6dfad90?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210104T213414Z&X-Amz-Expires=300&X-Amz-Signature=6f3c54211077092553590b33a7c36cd052895c9d4619607ad1df094782f64acf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=107914493&response-content-disposition=attachment%3B%20filename%3Ddatasette-0.53-macos-binary.zip&response-content-type=application%2Foctet-stream [following]\r\n--2021-01-04 21:34:14-- https://github-production-release-asset-2e65be.s3.amazonaws.com/107914493/74658700-4e90-11eb-8f3b-ee77e6dfad90?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210104T213414Z&X-Amz-Expires=300&X-Amz-Signature=6f3c54211077092553590b33a7c36cd052895c9d4619607ad1df094782f64acf&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=107914493&response-content-disposition=attachment%3B%20filename%3Ddatasette-0.53-macos-binary.zip&response-content-type=application%2Foctet-stream\r\nResolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.43.164\r\nConnecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.43.164|:443... connected.\r\nHTTP request sent, awaiting response... 200 OK\r\nLength: 8297283 (7.9M) [application/octet-stream]\r\nSaving to: \u2018datasette-0.53-macos-binary.zip\u2019\r\n\r\ndatasette-0.53-maco 100%[===================>] 7.91M --.-KB/s in 0.1s \r\n\r\n2021-01-04 21:34:14 (73.4 MB/s) - \u2018datasette-0.53-macos-binary.zip\u2019 saved [8297283/8297283]\r\n\r\nMac-1609795972770:tmp runner$ unzip datasette-0.53-macos-binary.zip \r\nArchive: datasette-0.53-macos-binary.zip\r\n creating: datasette-0.53-macos-binary/\r\n inflating: datasette-0.53-macos-binary/datasette \r\nMac-1609795972770:tmp runner$ datasette-0.53-macos-binary/datasette --help\r\nUsage: datasette [OPTIONS] COMMAND [ARGS]...\r\n\r\n Datasette!\r\n\r\nOptions:\r\n --version Show the version and exit.\r\n --help Show this message and exit.\r\n\r\nCommands:\r\n serve* Serve up specified SQLite database files with a web UI\r\n inspect\r\n install Install Python packages - e.g.\r\n package Package specified SQLite files into a new datasette Docker...\r\n plugins List currently available plugins\r\n publish Publish specified SQLite database files to the internet along...\r\n uninstall Uninstall Python packages (e.g.\r\nMac-1609795972770:tmp runner$ datasette-0.53-macos-binary/datasette --get /-/versions.json\r\n{\"python\": {\"version\": \"3.9.1\", \"full\": \"3.9.1 (default, Dec 10 2020, 10:36:35) \\n[Clang 12.0.0 (clang-1200.0.32.27)]\"}, \"datasette\": {\"version\": \"0.53\"}, \"asgi\": \"3.0\", \"uvicorn\": \"0.13.3\", \"sqlite\": {\"version\": \"3.34.0\", \"fts_versions\": [\"FTS5\", \"FTS4\", \"FTS3\"], \"extensions\": {\"json1\": null}, \"compile_options\": [\"COMPILER=clang-12.0.0\", \"ENABLE_COLUMN_METADATA\", \"ENABLE_FTS3\", \"ENABLE_FTS3_PARENTHESIS\", \"ENABLE_FTS4\", \"ENABLE_FTS5\", \"ENABLE_GEOPOLY\", \"ENABLE_JSON1\", \"ENABLE_PREUPDATE_HOOK\", \"ENABLE_RTREE\", \"ENABLE_SESSION\", \"MAX_VARIABLE_NUMBER=250000\", \"THREADSAFE=1\"]}}\r\nMac-1609795972770:tmp runner$ \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 273944952, "label": "Package as standalone binary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/93#issuecomment-754229977", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/93", "id": 754229977, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDIyOTk3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T21:28:01Z", "updated_at": "2021-01-04T21:28:01Z", "author_association": "OWNER", "body": "As an experiment, I put the macOS one in a zip file and attached it to the latest release:\r\n\r\n```\r\nmkdir datasette-0.53-macos-binary\r\ncp dist/datasette datasette-0.53-macos-binary\r\nzip -r datasette-0.53-macos-binary.zip datasette-0.53-macos-binary\r\n```\r\n\r\nIt's available here: https://github.com/simonw/datasette/releases/tag/0.53 - download URL is https://github.com/simonw/datasette/releases/download/0.53/datasette-0.53-macos-binary.zip", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 273944952, "label": "Package as standalone binary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/93#issuecomment-754227543", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/93", "id": 754227543, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDIyNzU0Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T21:23:13Z", "updated_at": "2021-01-04T21:23:13Z", "author_association": "OWNER", "body": "```\r\n(pyinstaller-venv) root@dogsheep:/tmp/pyinstaller-venv# dist/datasette --get /-/databases.json\r\n[{\"name\": \":memory:\", \"path\": null, \"size\": 0, \"is_mutable\": true, \"is_memory\": true, \"hash\": null}]\r\n(pyinstaller-venv) root@dogsheep:/tmp/pyinstaller-venv# ls -lah dist/datasette \r\n-rwxr-xr-x 1 root root 8.9M Jan 4 21:05 dist/datasette\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 273944952, "label": "Package as standalone binary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/93#issuecomment-754219002", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/93", "id": 754219002, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDIxOTAwMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T21:06:49Z", "updated_at": "2021-01-04T21:22:27Z", "author_association": "OWNER", "body": "Works on Linux/Ubuntu too, except I had to do `export BASE=` on a separate line. I also did this:\r\n```\r\napt-get install python3 python3-venv\r\npython3 -m venv pyinstaller-venv\r\nsource pyinstaller-venv/bin/activate\r\npip install wheel\r\npip install datasette pyinstaller\r\n\r\nexport DATASETTE_BASE=$(python -c 'import os; print(os.path.dirname(__import__(\"datasette\").__file__))')\r\n\r\npyinstaller -F \\\r\n --add-data \"$DATASETTE_BASE/templates:datasette/templates\" \\\r\n --add-data \"$DATASETTE_BASE/static:datasette/static\" \\\r\n --hidden-import datasette.publish \\\r\n --hidden-import datasette.publish.heroku \\\r\n --hidden-import datasette.publish.cloudrun \\\r\n --hidden-import datasette.facets \\\r\n --hidden-import datasette.sql_functions \\\r\n --hidden-import datasette.actor_auth_cookie \\\r\n --hidden-import datasette.default_permissions \\\r\n --hidden-import datasette.default_magic_parameters \\\r\n --hidden-import datasette.blob_renderer \\\r\n --hidden-import datasette.default_menu_links \\\r\n --hidden-import uvicorn \\\r\n --hidden-import uvicorn.logging \\\r\n --hidden-import uvicorn.loops \\\r\n --hidden-import uvicorn.loops.auto \\\r\n --hidden-import uvicorn.protocols \\\r\n --hidden-import uvicorn.protocols.http \\\r\n --hidden-import uvicorn.protocols.http.auto \\\r\n --hidden-import uvicorn.protocols.websockets \\\r\n --hidden-import uvicorn.protocols.websockets.auto \\\r\n --hidden-import uvicorn.lifespan \\\r\n --hidden-import uvicorn.lifespan.on \\\r\n $(which datasette)\r\n```\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 273944952, "label": "Package as standalone binary"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/93#issuecomment-754218545", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/93", "id": 754218545, "node_id": "MDEyOklzc3VlQ29tbWVudDc1NDIxODU0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-01-04T21:05:57Z", "updated_at": "2021-01-04T21:05:57Z", "author_association": "OWNER", "body": "That BASE= trick seems to work with `zsh` but not with `bash`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 273944952, "label": "Package as standalone binary"}, "performed_via_github_app": null}