{"html_url": "https://github.com/simonw/datasette/issues/236#issuecomment-645067611", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/236", "id": 645067611, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2NzYxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:50:12Z", "updated_at": "2020-06-16T23:50:59Z", "author_association": "OWNER", "body": "As for your other questions:\r\n\r\n> 1. I assume the goal is to have a CORS-friendly HTTPS endpoint that hosts the datasette service + user's db.\r\n\r\nYes, exactly. I know this will limit the size of database that can be deployed (since Lambda has a 50MB total package limit as far as I can tell) but there are plenty of interesting databases that are small enough to fit there.\r\n\r\nThe new EFS support for Lambda means that theoretically the size of database is now unlimited, which is really interesting. That's what got me inspired to take a look at a proof of concept in #850.\r\n\r\n> 2. If that's the goal, I think Lambda alone is insufficient. Lambda provides the compute fabric, but not the HTTP routing. You'd also need to add Application Load Balancer or API Gateway to provide an HTTP endpoint that routes to the lambda function.\r\n> \r\n> Do you have a preference between ALB or API GW? ALB has better economics at scale, but has a minimum monthly cost. API GW has worse per-request economics, but scales to zero when no requests are happening.\r\n\r\nI personally like scale-to-zero because many of my projects are likely to receive very little traffic. So API GW first, and maybe ALB as an option later on for people operating at scale?\r\n\r\n> 3. Does Datasette have any native components, or is it all pure python? If it has native bits, they'll likely need to be recompiled to work on Amazon Linux 2.\r\n\r\nAs you've found, the only native component is uvloop which is only needed if uvicorn is being used to serve requests.\r\n\r\n> 4. There are a few disparate services that need to be wired together to expose a Python service securely to the web. If I was doing this outside of the datasette publish system, I'd use an AWS CloudFormation template. Even within datasette, I think it still makes sense to use a CloudFormation template and just have the publish plugin invoke it (via the standard `aws` cli) with user-specified parameters. Does that sound reasonable to you?\r\n\r\nFor the eventual \"datasette publish lambda\" command I want whatever results in the smallest amount of inconvenience for users. I've been trying out Amazon SAM in #850 and it requires users to run Docker on their machines, which is a pretty huge barrier to entry! I don't have much experience with CloudFormation but it's probably a better bet, especially if you can \"pip install\" the dependencies needed to deploy with it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 317001500, "label": "datasette publish lambda plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/236#issuecomment-645066486", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/236", "id": 645066486, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2NjQ4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:45:45Z", "updated_at": "2020-06-16T23:45:45Z", "author_association": "OWNER", "body": "Hi Colin,\r\n\r\nSorry I didn't see this sooner! I've just started digging into this myself, to try and play with the new EFS Lambda support: #850.\r\n\r\nYes, uvloop is only needed because of uvicorn. I have a branch here that removes that dependency just for trying out Lambda: https://github.com/simonw/datasette/tree/no-uvicorn - so you can run `pip install https://github.com/simonw/datasette/archive/no-uvicorn.zip` to get that.\r\n\r\nI'm going to try out your `datasette-lambda` project next - really excited to see how far you've got with it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 317001500, "label": "datasette publish lambda plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645064332", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645064332, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2NDMzMg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:37:34Z", "updated_at": "2020-06-16T23:37:34Z", "author_association": "OWNER", "body": "Just realized Colin Dellow reported an issue with Datasette and Mangum back in April - #719 - and has in fact been working on https://github.com/code402/datasette-lambda for a while!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645063386", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645063386, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2MzM4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:34:07Z", "updated_at": "2020-06-16T23:34:07Z", "author_association": "OWNER", "body": "Tried `sam local invoke`:\r\n```\r\nsimon@Simons-MacBook-Pro datasette-proof-of-concept % sam local invoke\r\nInvoking app.lambda_handler (python3.8)\r\n\r\nFetching lambci/lambda:python3.8 Docker container image......\r\nMounting /private/tmp/datasette-proof-of-concept/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container\r\nSTART RequestId: 7c04480b-5d42-168e-dec0-4e8bf34fa596 Version: $LATEST\r\n[INFO]\t2020-06-16T23:33:27.24Z\t7c04480b-5d42-168e-dec0-4e8bf34fa596\tWaiting for application startup.\r\n[INFO]\t2020-06-16T23:33:27.24Z\t7c04480b-5d42-168e-dec0-4e8bf34fa596\tLifespanCycleState.STARTUP: 'lifespan.startup.complete' event received from application.\r\n[INFO]\t2020-06-16T23:33:27.24Z\t7c04480b-5d42-168e-dec0-4e8bf34fa596\tApplication startup complete.\r\n[INFO]\t2020-06-16T23:33:27.24Z\t7c04480b-5d42-168e-dec0-4e8bf34fa596\tWaiting for application shutdown.\r\n[INFO]\t2020-06-16T23:33:27.24Z\t7c04480b-5d42-168e-dec0-4e8bf34fa596\tLifespanCycleState.SHUTDOWN: 'lifespan.shutdown.complete' event received from application.\r\n[ERROR] KeyError: 'requestContext'\r\nTraceback (most recent call last):\r\n\u00a0\u00a0File \"/var/task/mangum/adapter.py\", line 110, in __call__\r\n\u00a0\u00a0\u00a0\u00a0return self.handler(event, context)\r\n\u00a0\u00a0File \"/var/task/mangum/adapter.py\", line 130, in handler\r\n\u00a0\u00a0\u00a0\u00a0if \"eventType\" in event[\"requestContext\"]:\r\nEND RequestId: 7c04480b-5d42-168e-dec0-4e8bf34fa596\r\nREPORT RequestId: 7c04480b-5d42-168e-dec0-4e8bf34fa596\tInit Duration: 1120.76 ms\tDuration: 7.08 ms\tBilled Duration: 100 ms\tMemory Size: 128 MBMax Memory Used: 47 MB\t\r\n\r\n{\"errorType\":\"KeyError\",\"errorMessage\":\"'requestContext'\",\"stackTrace\":[\" File \\\"/var/task/mangum/adapter.py\\\", line 110, in __call__\\n return self.handler(event, context)\\n\",\" File \\\"/var/task/mangum/adapter.py\\\", line 130, in handler\\n if \\\"eventType\\\" in event[\\\"requestContext\\\"]:\\n\"]}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645062266", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645062266, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2MjI2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:30:12Z", "updated_at": "2020-06-16T23:33:12Z", "author_association": "OWNER", "body": "OK, changed `requirements.txt` to this:\r\n```\r\nhttps://github.com/simonw/datasette/archive/no-uvicorn.zip\r\nmangum\r\n```\r\nNo `sam build --use-container` runs without errors. Ran `sam deploy` too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645063058", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645063058, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2MzA1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:32:57Z", "updated_at": "2020-06-16T23:32:57Z", "author_association": "OWNER", "body": "https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/ is now giving me a 500 internal server error.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645061088", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645061088, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2MTA4OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:25:41Z", "updated_at": "2020-06-16T23:25:41Z", "author_association": "OWNER", "body": "Someone else ran into this problem: https://github.com/iwpnd/fastapi-aws-lambda-example/issues/1\r\n\r\nSo I need to be able to pip install MOST of Datasette, but skip `uvicorn`. Tricky. I'll try installing a custom fork?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645060598", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645060598, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA2MDU5OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:24:01Z", "updated_at": "2020-06-16T23:24:01Z", "author_association": "OWNER", "body": "I changed `requirements.txt` to this:\r\n```\r\ndatasette\r\nmangum\r\n```\r\nAnd `app.py` to this:\r\n```python\r\nfrom datasette.app import Datasette\r\nfrom mangum import Mangum\r\n\r\n\r\ndatasette = Datasette([], memory=True)\r\nlambda_handler = Mangum(datasette.app())\r\n```\r\nBut then when I ran `sam build --use-container` I got this:\r\n```\r\nsimon@Simons-MacBook-Pro datasette-proof-of-concept % sam build --use-container\r\nStarting Build inside a container\r\nBuilding function 'HelloWorldFunction'\r\n\r\nFetching lambci/lambda:build-python3.8 Docker container image......\r\nMounting /private/tmp/datasette-proof-of-concept/hello_world as /tmp/samcli/source:ro,delegated inside runtime container\r\n\r\nBuild Failed\r\nRunning PythonPipBuilder:ResolveDependencies\r\nError: PythonPipBuilder:ResolveDependencies - {uvloop==0.14.0(wheel)}\r\n```\r\n`uvloop` isn't actually necessary for this project, since it's used by `uvicorn` which isn't needed if Lambda is serving ASGI traffic directly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645059663", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645059663, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1OTY2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:20:46Z", "updated_at": "2020-06-16T23:20:46Z", "author_association": "OWNER", "body": "I added an exclamation mark to hello world and ran `sam deploy` again. https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/ still shows the old message.\r\n\r\nRunning `sam build --use-container` first and then `sam deploy` did the right thing.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645058947", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645058947, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1ODk0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:18:18Z", "updated_at": "2020-06-16T23:18:18Z", "author_association": "OWNER", "body": "https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/\r\n\r\nThat's a pretty ugly URL. I'm not sure how to get rid of the `/Prod/` prefix on it. Might have to use the `base_url` setting to get something working: https://datasette.readthedocs.io/en/stable/config.html#base-url ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645058617", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645058617, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1ODYxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:17:09Z", "updated_at": "2020-06-16T23:17:09Z", "author_association": "OWNER", "body": "OK, `sam deploy --guided` now works!\r\n```\r\nsimon@Simons-MacBook-Pro datasette-proof-of-concept % sam deploy --guided\r\n\r\nConfiguring SAM deploy\r\n======================\r\n\r\n\tLooking for samconfig.toml : Not found\r\n\r\n\tSetting default arguments for 'sam deploy'\r\n\t=========================================\r\n\tStack Name [sam-app]: datasette-proof-of-concept\r\n\tAWS Region [us-east-1]: \r\n\t#Shows you resources changes to be deployed and require a 'Y' to initiate deploy\r\n\tConfirm changes before deploy [y/N]: \r\n\t#SAM needs permission to be able to create roles to connect to the resources in your template\r\n\tAllow SAM CLI IAM role creation [Y/n]: \r\n\tHelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y\r\n\tSave arguments to samconfig.toml [Y/n]: \r\n\r\n\tLooking for resources needed for deployment: Not found.\r\n\tCreating the required resources...\r\n\tSuccessfully created!\r\n\r\n\t\tManaged S3 bucket: aws-sam-cli-managed-default-samclisourcebucket-1ksajo4h62s07\r\n\t\tA different default S3 bucket can be set in samconfig.toml\r\n\r\n\tSaved arguments to config file\r\n\tRunning 'sam deploy' for future deployments will use the parameters saved above.\r\n\tThe above parameters can be changed by modifying samconfig.toml\r\n\tLearn more about samconfig.toml syntax at \r\n\thttps://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-config.html\r\n\r\n\tDeploying with following values\r\n\t===============================\r\n\tStack name : datasette-proof-of-concept\r\n\tRegion : us-east-1\r\n\tConfirm changeset : False\r\n\tDeployment s3 bucket : aws-sam-cli-managed-default-samclisourcebucket-1ksajo4h62s07\r\n\tCapabilities : [\"CAPABILITY_IAM\"]\r\n\tParameter overrides : {}\r\n\r\nInitiating deployment\r\n=====================\r\nUploading to datasette-proof-of-concept/0c208b5656a7aeb6186d49bebc595237 535344 / 535344.0 (100.00%)\r\nHelloWorldFunction may not have authorization defined.\r\nUploading to datasette-proof-of-concept/14bd9ce3e21f9c88634d13c0c9b377e4.template 1147 / 1147.0 (100.00%)\r\n\r\nWaiting for changeset to be created..\r\n\r\nCloudFormation stack changeset\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\nOperation LogicalResourceId ResourceType \r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\n+ Add HelloWorldFunctionHelloWorldPermissionProd AWS::Lambda::Permission \r\n+ Add HelloWorldFunctionRole AWS::IAM::Role \r\n+ Add HelloWorldFunction AWS::Lambda::Function \r\n+ Add ServerlessRestApiDeployment47fc2d5f9d AWS::ApiGateway::Deployment \r\n+ Add ServerlessRestApiProdStage AWS::ApiGateway::Stage \r\n+ Add ServerlessRestApi AWS::ApiGateway::RestApi \r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\n\r\nChangeset created successfully. arn:aws:cloudformation:us-east-1:462092780466:changeSet/samcli-deploy1592349262/d685f2de-87c1-4b8e-b13a-67b94f8fc928\r\n\r\n\r\n2020-06-16 16:14:29 - Waiting for stack create/update to complete\r\n\r\nCloudFormation events from changeset\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\nResourceStatus ResourceType LogicalResourceId ResourceStatusReason \r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\nCREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole - \r\nCREATE_IN_PROGRESS AWS::IAM::Role HelloWorldFunctionRole Resource creation Initiated \r\nCREATE_COMPLETE AWS::IAM::Role HelloWorldFunctionRole - \r\nCREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction Resource creation Initiated \r\nCREATE_IN_PROGRESS AWS::Lambda::Function HelloWorldFunction - \r\nCREATE_COMPLETE AWS::Lambda::Function HelloWorldFunction - \r\nCREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi Resource creation Initiated \r\nCREATE_IN_PROGRESS AWS::ApiGateway::RestApi ServerlessRestApi - \r\nCREATE_COMPLETE AWS::ApiGateway::RestApi ServerlessRestApi - \r\nCREATE_IN_PROGRESS AWS::Lambda::Permission HelloWorldFunctionHelloWorldPermissi - \r\n onProd \r\nCREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc2d5f9 - \r\n d \r\nCREATE_COMPLETE AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc2d5f9 - \r\n d \r\nCREATE_IN_PROGRESS AWS::ApiGateway::Deployment ServerlessRestApiDeployment47fc2d5f9 Resource creation Initiated \r\n d \r\nCREATE_IN_PROGRESS AWS::Lambda::Permission HelloWorldFunctionHelloWorldPermissi Resource creation Initiated \r\n onProd \r\nCREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage - \r\nCREATE_COMPLETE AWS::ApiGateway::Stage ServerlessRestApiProdStage - \r\nCREATE_IN_PROGRESS AWS::ApiGateway::Stage ServerlessRestApiProdStage Resource creation Initiated \r\nCREATE_COMPLETE AWS::Lambda::Permission HelloWorldFunctionHelloWorldPermissi - \r\n onProd \r\nCREATE_COMPLETE AWS::CloudFormation::Stack datasette-proof-of-concept - \r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\n\r\nCloudFormation outputs from deployed stack\r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\nOutputs \r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\nKey HelloWorldFunctionIamRole \r\nDescription Implicit IAM Role created for Hello World function \r\nValue arn:aws:iam::462092780466:role/datasette-proof-of-concept-HelloWorldFunctionRole-8MIDNIV5ECA6 \r\n\r\nKey HelloWorldApi \r\nDescription API Gateway endpoint URL for Prod stage for Hello World function \r\nValue https://q7lymja3sj.execute-api.us-east-1.amazonaws.com/Prod/hello/ \r\n\r\nKey HelloWorldFunction \r\nDescription Hello World Lambda Function ARN \r\nValue arn:aws:lambda:us-east-1:462092780466:function:datasette-proof-of-concept-HelloWorldFunction-QTF78ZEUDCB \r\n---------------------------------------------------------------------------------------------------------------------------------------------------------\r\n\r\nSuccessfully created/updated stack - datasette-proof-of-concept in us-east-1\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645056636", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645056636, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1NjYzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:10:22Z", "updated_at": "2020-06-16T23:10:22Z", "author_association": "OWNER", "body": "Clicking that button generated me an access key ID / access key secret pair. Dropping those into `~/.aws/credentials` using this format:\r\n```\r\n[default]\r\naws_access_key_id = your_access_key_id\r\naws_secret_access_key = your_secret_access_key\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645055200", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645055200, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1NTIwMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:05:48Z", "updated_at": "2020-06-16T23:05:48Z", "author_association": "OWNER", "body": "Logged in as `simon-administrator` I'm using https://console.aws.amazon.com/iam/home?region=us-east-2#/security_credentials to create credentials:\r\n\r\n\"Banners_and_Alerts_and_IAM_Management_Console\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645054206", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645054206, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1NDIwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:02:54Z", "updated_at": "2020-06-16T23:04:59Z", "author_association": "OWNER", "body": "I think I need to sign in to the AWS console with this new `simon-administrator` account and create IAM credentials for it.\r\n\r\n... for which I needed my root \"account ID\" - a 12 digit number - to use on the IAM login form.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645053923", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645053923, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1MzkyMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T23:01:49Z", "updated_at": "2020-06-16T23:01:49Z", "author_association": "OWNER", "body": "I used https://console.aws.amazon.com/billing/home?#/account and activated \"IAM user/role access to billing information\" - what a puzzling first step!\r\n\r\nI created a new user with AWS console access (which means access to the web UI) called `simon-administrator` and set a password. I created an `Administrators` group with `AdministratorAccess`.\r\n\r\n\"Banners_and_Alerts_and_IAM_Management_Console\"\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645051972", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645051972, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1MTk3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:55:04Z", "updated_at": "2020-06-16T22:55:04Z", "author_association": "OWNER", "body": "```\r\nsimon@Simons-MacBook-Pro datasette-proof-of-concept % sam deploy --guided\r\n\r\nConfiguring SAM deploy\r\n======================\r\n\r\n\tLooking for samconfig.toml : Not found\r\n\r\n\tSetting default arguments for 'sam deploy'\r\n\t=========================================\r\n\tStack Name [sam-app]: datasette-proof-of-concept\r\n\tAWS Region [us-east-1]: \r\n\t#Shows you resources changes to be deployed and require a 'Y' to initiate deploy\r\n\tConfirm changes before deploy [y/N]: y\r\n\t#SAM needs permission to be able to create roles to connect to the resources in your template\r\n\tAllow SAM CLI IAM role creation [Y/n]: y\r\n\tHelloWorldFunction may not have authorization defined, Is this okay? [y/N]: y\r\n\tSave arguments to samconfig.toml [Y/n]: y\r\nError: Failed to create managed resources: Unable to locate credentials\r\n```\r\nI need to get my AWS credentials sorted. I'm going to follow https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html and https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started-set-up-credentials.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645051370", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645051370, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1MTM3MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:53:05Z", "updated_at": "2020-06-16T22:53:05Z", "author_association": "OWNER", "body": "```\r\nsimon@Simons-MacBook-Pro datasette-proof-of-concept % sam local invoke\r\nInvoking app.lambda_handler (python3.8)\r\n\r\nFetching lambci/lambda:python3.8 Docker container image....................................................................................................................................................................................................................................\r\nMounting /private/tmp/datasette-proof-of-concept/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container\r\nSTART RequestId: 4616ab43-6882-1627-e5e3-5a29730d52f9 Version: $LATEST\r\nEND RequestId: 4616ab43-6882-1627-e5e3-5a29730d52f9\r\nREPORT RequestId: 4616ab43-6882-1627-e5e3-5a29730d52f9\tInit Duration: 140.84 ms\tDuration: 2.49 ms\tBilled Duration: 100 ms\tMemory Size: 128 MBMax Memory Used: 25 MB\t\r\n\r\n{\"statusCode\":200,\"body\":\"{\\\"message\\\": \\\"hello world\\\"}\"}\r\nsimon@Simons-MacBook-Pro datasette-proof-of-concept % sam local invoke\r\nInvoking app.lambda_handler (python3.8)\r\n\r\nFetching lambci/lambda:python3.8 Docker container image......\r\nMounting /private/tmp/datasette-proof-of-concept/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated inside runtime container\r\nSTART RequestId: 3189df2f-e9c0-1be4-b9ac-f329c5fcd067 Version: $LATEST\r\nEND RequestId: 3189df2f-e9c0-1be4-b9ac-f329c5fcd067\r\nREPORT RequestId: 3189df2f-e9c0-1be4-b9ac-f329c5fcd067\tInit Duration: 87.22 ms\tDuration: 2.34 ms\tBilled Duration: 100 ms\tMemory Size: 128 MB\tMax Memory Used: 25 MB\t\r\n\r\n{\"statusCode\":200,\"body\":\"{\\\"message\\\": \\\"hello world\\\"}\"}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645050948", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645050948, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA1MDk0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:51:30Z", "updated_at": "2020-06-16T22:52:30Z", "author_association": "OWNER", "body": "```\r\nsimon@Simons-MacBook-Pro datasette-proof-of-concept % sam build --use-container\r\nStarting Build inside a container\r\nBuilding function 'HelloWorldFunction'\r\n\r\nFetching lambci/lambda:build-python3.8 Docker container image..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................\r\nMounting /private/tmp/datasette-proof-of-concept/hello_world as /tmp/samcli/source:ro,delegated inside runtime container\r\n\r\nBuild Succeeded\r\n\r\nBuilt Artifacts : .aws-sam/build\r\nBuilt Template : .aws-sam/build/template.yaml\r\n\r\nCommands you can use next\r\n=========================\r\n[*] Invoke Function: sam local invoke\r\n[*] Deploy: sam deploy --guided\r\n \r\nRunning PythonPipBuilder:ResolveDependencies\r\nRunning PythonPipBuilder:CopySource\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645048062", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645048062, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA0ODA2Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:41:33Z", "updated_at": "2020-06-16T22:41:33Z", "author_association": "OWNER", "body": "```\r\nsimon@Simons-MacBook-Pro /tmp % sam init\r\n\r\n\tSAM CLI now collects telemetry to better understand customer needs.\r\n\r\n\tYou can OPT OUT and disable telemetry collection by setting the\r\n\tenvironment variable SAM_CLI_TELEMETRY=0 in your shell.\r\n\tThanks for your help!\r\n\r\n\tLearn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html\r\n\r\nWhich template source would you like to use?\r\n\t1 - AWS Quick Start Templates\r\n\t2 - Custom Template Location\r\nChoice: 1\r\n\r\nWhich runtime would you like to use?\r\n\t1 - nodejs12.x\r\n\t2 - python3.8\r\n\t3 - ruby2.7\r\n\t4 - go1.x\r\n\t5 - java11\r\n\t6 - dotnetcore3.1\r\n\t7 - nodejs10.x\r\n\t8 - python3.7\r\n\t9 - python3.6\r\n\t10 - python2.7\r\n\t11 - ruby2.5\r\n\t12 - java8\r\n\t13 - dotnetcore2.1\r\nRuntime: 2\r\n\r\nProject name [sam-app]: datasette-proof-of-concept\r\n\r\nCloning app templates from https://github.com/awslabs/aws-sam-cli-app-templates.git\r\n\r\nAWS quick start application templates:\r\n\t1 - Hello World Example\r\n\t2 - EventBridge Hello World\r\n\t3 - EventBridge App from scratch (100+ Event Schemas)\r\n\t4 - Step Functions Sample App (Stock Trader)\r\nTemplate selection: 1\r\n\r\n-----------------------\r\nGenerating application:\r\n-----------------------\r\nName: datasette-proof-of-concept\r\nRuntime: python3.8\r\nDependency Manager: pip\r\nApplication Template: hello-world\r\nOutput Directory: .\r\n\r\nNext steps can be found in the README file at ./datasette-proof-of-concept/README.md\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645047703", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645047703, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA0NzcwMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:40:19Z", "updated_at": "2020-06-16T22:40:19Z", "author_association": "OWNER", "body": "Installed SAM:\r\n```\r\nbrew tap aws/tap\r\nbrew install aws-sam-cli\r\nsam --version\r\nSAM CLI, version 0.52.0\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645045055", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645045055, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA0NTA1NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:31:49Z", "updated_at": "2020-06-16T22:31:49Z", "author_association": "OWNER", "body": "It looks like SAM - AWS Serverless Application Model - is the currently recommended way to deploy Python apps to Lambda from the command-line: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started-hello-world.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645042625", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645042625, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA0MjYyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:24:26Z", "updated_at": "2020-06-16T22:24:26Z", "author_association": "OWNER", "body": "From https://mangum.io/adapter/\r\n\r\n> The AWS Lambda handler `event` and `context` arguments are made available to an ASGI application in the ASGI connection scope.\r\n> \r\n> ```\r\n> scope['aws.event']\r\n> scope['aws.context']\r\n> ```\r\nI can use https://github.com/simonw/datasette-debug-asgi to see that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645041663", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645041663, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTA0MTY2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T22:21:44Z", "updated_at": "2020-06-16T22:21:44Z", "author_association": "OWNER", "body": "https://github.com/jordaneremieff/mangum looks like the best way to run an ASGI app on Lambda at the moment.\r\n\r\n```python\r\nfrom mangum import Mangum\r\n\r\nasync def app(scope, receive, send):\r\n await send(\r\n {\r\n \"type\": \"http.response.start\",\r\n \"status\": 200,\r\n \"headers\": [[b\"content-type\", b\"text/plain; charset=utf-8\"]],\r\n }\r\n )\r\n await send({\"type\": \"http.response.body\", \"body\": b\"Hello, world!\"})\r\n\r\n\r\nhandler = Mangum(app)\r\n``` ", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645032643", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645032643, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTAzMjY0Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T21:57:10Z", "updated_at": "2020-06-16T21:57:10Z", "author_association": "OWNER", "body": "https://docs.aws.amazon.com/efs/latest/ug/wt1-getting-started.html is an EFS walk-through using the AWS CLI tool instead of clicking around in their web interface.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645031225", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645031225, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTAzMTIyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T21:53:25Z", "updated_at": "2020-06-16T21:53:25Z", "author_association": "OWNER", "body": "Easier solution to this might be to have two functions - a \"read-only\" one which is allowed to scale as much as it likes, and a \"write-only\" one which can write to the database files but is limited to running a maximum of one Lambda instance. https://docs.aws.amazon.com/lambda/latest/dg/invocation-scaling.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/850#issuecomment-645030262", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/850", "id": 645030262, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NTAzMDI2Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T21:51:01Z", "updated_at": "2020-06-16T21:51:39Z", "author_association": "OWNER", "body": "File locking is interesting here. https://docs.aws.amazon.com/lambda/latest/dg/services-efs.html\r\n\r\n> Amazon EFS supports [file locking](https://docs.aws.amazon.com/efs/latest/ug/how-it-works.html#consistency) to prevent corruption if multiple functions try to write to the same file system at the same time. Locking in Amazon EFS follows the NFS v4.1 protocol for advisory locking, and enables your applications to use both whole file and byte range locks. \r\n\r\nSQLite can apparently work on NFS v4.1. I think I'd rather set things up so there's only ever one writer - so a Datasette instance could scale reads by running lots more lambda functions but only one function ever writes to a file at a time. Not sure if that's feasible with Lambda though - maybe by adding some additional shared state mechanism like Redis?", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639993467, "label": "Proof of concept for Datasette on AWS Lambda with EFS"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/690#issuecomment-644987083", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/690", "id": 644987083, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NDk4NzA4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T20:11:35Z", "updated_at": "2020-06-16T20:11:35Z", "author_association": "OWNER", "body": "Twitter conversation about drop-down menu solutions that are accessible, fast loading and use minimal JavaScript: https://twitter.com/simonw/status/1272974294545395712\r\n\r\nI _really_ like the approach taken by GitHub Primer, which builds on top of HTML `` `
` tags: https://primer.style/css/components/dropdown", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 573755726, "label": "Mechanism for plugins to add action menu items for various things"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/849#issuecomment-644584075", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/849", "id": 644584075, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NDU4NDA3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-16T07:24:08Z", "updated_at": "2020-06-16T07:24:08Z", "author_association": "OWNER", "body": "This guide is fantastic - I'll be following it closely: https://github.com/chancancode/branch-rename/blob/main/README.md - in particular the Action to mirror master and main for a while.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639072811, "label": "Rename master branch to main"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/849#issuecomment-644384787", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/849", "id": 644384787, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NDM4NDc4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-15T20:56:07Z", "updated_at": "2020-06-15T20:56:19Z", "author_association": "OWNER", "body": "The big question is how this impacts existing CI configuration. `datasette-psutil` is configured to use Circle CI, what happens if I push a new commit?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639072811, "label": "Rename master branch to main"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/849#issuecomment-644384417", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/849", "id": 644384417, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NDM4NDQxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-15T20:55:23Z", "updated_at": "2020-06-15T20:55:23Z", "author_association": "OWNER", "body": "I'm doing https://github.com/simonw/datasette-psutil first.\r\n\r\nIn my local checkout:\r\n```\r\ngit branch -m master main\r\ngit push -u origin main\r\n```\r\n(Thanks, https://www.hanselman.com/blog/EasilyRenameYourGitDefaultBranchFromMasterToMain.aspx)\r\n\r\nThen in https://github.com/simonw/datasette-psutil/settings/branches I changed the default branch to `main`.\r\n\r\n\"Branches\"\r\n\r\nLinks to these docs: https://help.github.com/en/github/administering-a-repository/setting-the-default-branch\r\n\r\nThat worked! https://github.com/simonw/datasette-psutil\r\n\r\nOne catch, which I think will impact my most widely used repos the most (like datasette) - linking to a specific file now looks like this:\r\n\r\nhttps://github.com/simonw/datasette-psutil/blob/main/datasette_psutil/__init__.py\r\n\r\nThe old https://github.com/simonw/datasette-psutil/blob/master/datasette_psutil/__init__.py link is presumably frozen in time?\r\n\r\nI've definitely got links spread around the web to my \"most recent version of this code\" that would use the `master` reference, which would need to be updated to `main` instead. Most of those are probably in the Datasette docs and on my blog though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639072811, "label": "Rename master branch to main"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/849#issuecomment-644322234", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/849", "id": 644322234, "node_id": "MDEyOklzc3VlQ29tbWVudDY0NDMyMjIzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-15T19:06:16Z", "updated_at": "2020-06-15T19:06:16Z", "author_association": "OWNER", "body": "I'll make this change on a few of my other repos first to make sure I haven't missed any tricky edge-cases.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 639072811, "label": "Rename master branch to main"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/691#issuecomment-643709037", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/691", "id": 643709037, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzcwOTAzNw==", "user": {"value": 49260, "label": "amjith"}, "created_at": "2020-06-14T02:35:16Z", "updated_at": "2020-06-14T02:35:16Z", "author_association": "CONTRIBUTOR", "body": "The server should reload in the `config_dir` mode. \r\n\r\nRef: #848", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 574021194, "label": "--reload sould reload server if code in --plugins-dir changes"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/847#issuecomment-643704730", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/847", "id": 643704730, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzcwNDczMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-14T01:28:34Z", "updated_at": "2020-06-14T01:28:34Z", "author_association": "OWNER", "body": "Here's the plugin that adds those custom SQLite functions:\r\n```python\r\nfrom datasette import hookimpl\r\nfrom coverage.numbits import register_sqlite_functions\r\n\r\n\r\n@hookimpl\r\ndef prepare_connection(conn):\r\n register_sqlite_functions(conn)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638259643, "label": "Take advantage of .coverage being a SQLite database"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/847#issuecomment-643704565", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/847", "id": 643704565, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzcwNDU2NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-14T01:26:56Z", "updated_at": "2020-06-14T01:26:56Z", "author_association": "OWNER", "body": "On closer inspection, I don't know if there's that much useful stuff you can do with the data from `.coverage` on its own.\r\n\r\nConsider the following query against a `.coverage` run against Datasette itself:\r\n\r\n```sql\r\nselect file_id, context_id, numbits_to_nums(numbits) from line_bits\r\n```\r\n\"_coverage__select_file_id__context_id__numbits_to_nums_numbits__from_line_bits\"\r\n\r\nIt looks like this tells me which lines of which files were executed during the test run. But... without the actual source code, I don't think I can calculate the coverage percentage for each file. I don't want to count comment lines or whitespace as untested for example, and I don't know how many lines were in the file.\r\n\r\nIf I'm right that it's not possible to calculate percentage coverage from just the `.coverage` data then I'll need to do something a bit more involved - maybe parsing the `coverage.xml` report and loading that into my own schema?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638259643, "label": "Take advantage of .coverage being a SQLite database"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/847#issuecomment-643702715", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/847", "id": 643702715, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzcwMjcxNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-14T01:03:30Z", "updated_at": "2020-06-14T01:03:40Z", "author_association": "OWNER", "body": "Filed a related issue with some ideas against `coveragepy` here: https://github.com/nedbat/coveragepy/issues/999", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638259643, "label": "Take advantage of .coverage being a SQLite database"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/846#issuecomment-643699583", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/846", "id": 643699583, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY5OTU4Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-14T00:26:31Z", "updated_at": "2020-06-14T00:26:31Z", "author_association": "OWNER", "body": "That seems to have fixed the problem, at least for the moment.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638241779, "label": "\"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/846#issuecomment-643699063", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/846", "id": 643699063, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY5OTA2Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-14T00:22:32Z", "updated_at": "2020-06-14T00:22:32Z", "author_association": "OWNER", "body": "Idea: `num_sql_threads` (described as \"Number of threads in the thread pool for executing SQLite queries\") defaults to 3 - can I knock that down to 1 in the tests and open less connections as a result?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638241779, "label": "\"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/846#issuecomment-643698790", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/846", "id": 643698790, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY5ODc5MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-14T00:20:42Z", "updated_at": "2020-06-14T00:20:42Z", "author_association": "OWNER", "body": "Released a new plugin, `datasette-psutil`, as a side-effect of this investigation: https://github.com/simonw/datasette-psutil", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638241779, "label": "\"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/846#issuecomment-643685669", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/846", "id": 643685669, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY4NTY2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T22:24:22Z", "updated_at": "2020-06-13T22:24:22Z", "author_association": "OWNER", "body": "I tried this experiment:\r\n```python\r\nimport sqlite3, psutil\r\ndef show_things():\r\n conn = sqlite3.connect(\"fixtures.db\")\r\n tables = [r[0] for r in conn.execute(\"select * from sqlite_master\").fetchall()]\r\n return tables\r\nprint(psutil.Process().open_files())\r\nprint(show_things())\r\nprint(psutil.Process().open_files())\r\n```\r\nTo see if the connection would be automatically released when the `conn` variable was garbage collected at the end of the function... and it was correctly released - the two calls to `open_files()` showed that the file did not remain open.\r\n\r\nLikewise:\r\n```\r\nIn [11]: conn = sqlite3.connect(\"fixtures.db\") \r\n\r\nIn [12]: psutil.Process().open_files() \r\nOut[12]: \r\n[popenfile(path='/Users/simon/.ipython/profile_default/history.sqlite', fd=4),\r\n popenfile(path='/Users/simon/.ipython/profile_default/history.sqlite', fd=5),\r\n popenfile(path='/Users/simon/Dropbox/Development/datasette/fixtures.db', fd=12)]\r\n\r\nIn [13]: del conn \r\n\r\nIn [14]: psutil.Process().open_files() \r\nOut[14]: \r\n[popenfile(path='/Users/simon/.ipython/profile_default/history.sqlite', fd=4),\r\n popenfile(path='/Users/simon/.ipython/profile_default/history.sqlite', fd=5)]\r\n```\r\nSo presumably there's something about the way my pytest fixtures work that's causing the many different `Datasette()` instances and their underlying SQLite connections that I create not to be cleaned up later.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638241779, "label": "\"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/846#issuecomment-643685333", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/846", "id": 643685333, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY4NTMzMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T22:19:38Z", "updated_at": "2020-06-13T22:19:38Z", "author_association": "OWNER", "body": "That's 91 open files but only 29 unique filenames.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638241779, "label": "\"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/846#issuecomment-643685207", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/846", "id": 643685207, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY4NTIwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T22:18:01Z", "updated_at": "2020-06-13T22:18:01Z", "author_association": "OWNER", "body": "This shows currently open files (after `pip install psutil`):\r\n```\r\nimport psutil\r\npsutil.Process().open_files()\r\n```\r\n\r\nI ran it inside `pytest -x --pdb` and got this:\r\n```\r\n> /Users/simon/.local/share/virtualenvs/datasette-AWNrQs95/lib/python3.7/site-packages/jinja2/utils.py(154)open_if_exists()\r\n-> return open(filename, mode)\r\n(Pdb) import psutil\r\n(Pdb) psutil.Process().open_files()\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp9uhx5d8x/fixtures.db', fd=10),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyfw44ica/fixtures.dot.db', fd=11),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyrg6g48b/fixtures.db', fd=12),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp33kkg62s/fixtures.db', fd=13),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp33kkg62s/fixtures.db', fd=14),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp33kkg62s/fixtures.db', fd=15),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp33kkg62s/fixtures.db', fd=16),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp33kkg62s/fixtures.db', fd=17),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp33kkg62s/fixtures.db', fd=18),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp33kkg62s/fixtures.db', fd=19),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpng4lg84_/fixtures.db', fd=20),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp9uhx5d8x/fixtures.db', fd=21),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp9uhx5d8x/fixtures.db', fd=22),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp9uhx5d8x/fixtures.db', fd=23),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmph11oalw_/fixtures.db', fd=24),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyfw44ica/fixtures.dot.db', fd=25),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyfw44ica/fixtures.dot.db', fd=26),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyfw44ica/fixtures.dot.db', fd=27),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpiorb2bo9/fixtures.db', fd=28),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyrg6g48b/fixtures.db', fd=29),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyrg6g48b/fixtures.db', fd=30),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpyrg6g48b/fixtures.db', fd=31),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmprvyj5udv/fixtures.db', fd=32),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpng4lg84_/fixtures.db', fd=33),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpng4lg84_/fixtures.db', fd=34),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpng4lg84_/fixtures.db', fd=35),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpb_l6gmq0/fixtures.db', fd=36),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmph11oalw_/extra database.db', fd=40),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpf0py4thp/fixtures.db', fd=41),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpiorb2bo9/fixtures.db', fd=42),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpiorb2bo9/fixtures.db', fd=43),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpiorb2bo9/fixtures.db', fd=44),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmph11oalw_/fixtures.db', fd=45),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmph11oalw_/fixtures.db', fd=52),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/fixtures.db', fd=53),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmprvyj5udv/fixtures.db', fd=54),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmprvyj5udv/fixtures.db', fd=55),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmprvyj5udv/fixtures.db', fd=56),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpoveuwqn6/fixtures.db', fd=57),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpb_l6gmq0/fixtures.db', fd=61),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpb_l6gmq0/fixtures.db', fd=62),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpb_l6gmq0/fixtures.db', fd=63),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp_j4h9mrn/fixtures.db', fd=64),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpf0py4thp/fixtures.db', fd=65),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpf0py4thp/fixtures.db', fd=66),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpf0py4thp/extra database.db', fd=67),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpf0py4thp/extra database.db', fd=68),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpf0py4thp/fixtures.db', fd=69),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpf0py4thp/extra database.db', fd=70)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpub3eodj1/fixtures.db', fd=71)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/fixtures.db', fd=72)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/foo.db', fd=73)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/foo.db', fd=74)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/fixtures.db', fd=75)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/fixtures.db', fd=76)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/foo.db', fd=77)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/foo-bar.db', fd=78)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/foo-bar.db', fd=79)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpwgcnmg4b/foo-bar.db', fd=80)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/pytest-of-simon/pytest-4/config-dir0/immutable.db', fd=81),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpoveuwqn6/fixtures.db', fd=82),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpoveuwqn6/fixtures.db', fd=83),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpoveuwqn6/fixtures.db', fd=84),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp44w5d5wo/fixtures.db', fd=85),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp_j4h9mrn/fixtures.db', fd=86),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp_j4h9mrn/fixtures.db', fd=87),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp_j4h9mrn/fixtures.db', fd=88),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpvu7h14uy/fixtures.db', fd=89),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/pytest-of-simon/pytest-4/config-dir0/demo.db', fd=119),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/pytest-of-simon/pytest-4/config-dir0/demo.db', fd=120),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/pytest-of-simon/pytest-4/config-dir0/demo.db', fd=121),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp0xcnrjag/fixtures.db', fd=122),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpub3eodj1/fixtures.db', fd=123),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpub3eodj1/fixtures.db', fd=124),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpub3eodj1/fixtures.db', fd=125),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpfz8go8rk/fixtures.db', fd=126),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp44w5d5wo/fixtures.db', fd=127),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp44w5d5wo/fixtures.db', fd=128),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp44w5d5wo/fixtures.db', fd=129),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp5j3k1ep_/fixtures.db', fd=130)\r\n popenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpvu7h14uy/fixtures.db', fd=131),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpvu7h14uy/fixtures.db', fd=132),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpvo3cobk9/fixtures.db', fd=133),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp2t9txyir/fixtures.db', fd=134),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpfz8go8rk/fixtures.db', fd=135),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmpfz8go8rk/fixtures.db', fd=136),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp7h3skv8b/fixtures.db', fd=137),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp5j3k1ep_/fixtures.db', fd=138),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp5j3k1ep_/fixtures.db', fd=139),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp5j3k1ep_/fixtures.db', fd=140),\r\npopenfile(path='/private/var/folders/wr/hn3206rs1yzgq3r49bz8nvnh0000gn/T/tmp5j3k1ep_/extra database.db', fd=141),\r\n```\r\nSo yeah, that's too many open files!\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638241779, "label": "\"Too many open files\" error running tests"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643681747", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643681747, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY4MTc0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T21:38:46Z", "updated_at": "2020-06-13T21:38:46Z", "author_association": "OWNER", "body": "Closing this because I've researched feasibility. I may start a milestone in the future to help me get to 100%.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/844#issuecomment-643681517", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/844", "id": 643681517, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY4MTUxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T21:36:15Z", "updated_at": "2020-06-13T21:36:15Z", "author_association": "OWNER", "body": "OK, this works now: https://codecov.io/gh/simonw/datasette/tree/1210d9f41841bdca450f85a2342cdb0ff339c1b4", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638230433, "label": "Action to run tests and upload coverage report"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/843#issuecomment-643676314", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/843", "id": 643676314, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY3NjMxNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T20:47:37Z", "updated_at": "2020-06-13T20:47:37Z", "author_association": "OWNER", "body": "I can use this action: https://github.com/codecov/codecov-action", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638229448, "label": "Configure codecov.io"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/843#issuecomment-643676069", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/843", "id": 643676069, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY3NjA2OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T20:45:29Z", "updated_at": "2020-06-13T20:45:29Z", "author_association": "OWNER", "body": "I set up https://codecov.io/gh/simonw/datasette/settings and added a `CODECOV_TOKEN` to the GitHub Actions secrets for this repo.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638229448, "label": "Configure codecov.io"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/842#issuecomment-643663005", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/842", "id": 643663005, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY2MzAwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T18:51:57Z", "updated_at": "2020-06-13T18:51:57Z", "author_association": "OWNER", "body": "Two potential designs:\r\n\r\n- `_actor_id`, `_request_ip`, `_now_timestamp` - so special reserved parameters\r\n- a SQL function: `update blah set up = special('ip')`\r\n\r\nI fee the first would be easier to implement.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638212085, "label": "Magic parameters for canned queries"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643661125", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643661125, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY2MTEyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T18:35:30Z", "updated_at": "2020-06-13T18:36:50Z", "author_association": "OWNER", "body": "I ran export CODECOV_TOKEN=\"f7935cad...\", then ran this:\r\n```\r\ndatasette $ bash <(curl -s https://codecov.io/bash) \r\n _____ _\r\n / ____| | |\r\n| | ___ __| | ___ ___ _____ __\r\n| | / _ \\ / _` |/ _ \\/ __/ _ \\ \\ / /\r\n| |___| (_) | (_| | __/ (_| (_) \\ V /\r\n \\_____\\___/ \\__,_|\\___|\\___\\___/ \\_/\r\n Bash-20200602-f809a24\r\n\r\n\r\nx> No CI provider detected.\r\n Testing inside Docker? http://docs.codecov.io/docs/testing-with-docker\r\n Testing with Tox? https://docs.codecov.io/docs/python#section-testing-with-tox\r\n project root: .\r\n--> token set from env\r\n Yaml not found, that's ok! Learn more at http://docs.codecov.io/docs/codecov-yaml\r\n==> Running gcov in . (disable via -X gcov)\r\n==> Searching for coverage reports in:\r\n + .\r\n -> Found 1 reports\r\n==> Detecting git/mercurial file structure\r\n==> Reading reports\r\n + ./coverage.xml bytes=139174\r\n==> Appending adjustments\r\n https://docs.codecov.io/docs/fixing-reports\r\n -> No adjustments found\r\n==> Gzipping contents\r\n==> Uploading reports\r\n url: https://codecov.io\r\n query: branch=master&commit=0e49842e227a0f1f69d48108c87d17fe0379e548&build=&build_url=&name=&tag=&slug=simonw%2Fdatasette&service=&flags=&pr=&job=\r\n -> Pinging Codecov\r\nhttps://codecov.io/upload/v4?package=bash-20200602-f809a24&token=secret&branch=master&commit=0e49842e227a0f1f69d48108c87d17fe0379e548&build=&build_url=&name=&tag=&slug=simonw%2Fdatasette&service=&flags=&pr=&job=\r\n -> Uploading\r\n -> View reports at https://codecov.io/github/simonw/datasette/commit/0e49842e227a0f1f69d48108c87d17fe0379e548\r\n```\r\nBut https://codecov.io/github/simonw/datasette/commit/0e49842e227a0f1f69d48108c87d17fe0379e548 is a 404, so it doesn't seem to have worked?\r\n\r\nUPDATE: It works now, took about 30 seconds before the report showed up at that URL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643660757", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643660757, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY2MDc1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T18:32:20Z", "updated_at": "2020-06-13T18:32:20Z", "author_association": "OWNER", "body": "Looking at options for publishing coverage reports:\r\n\r\n* https://github.com/codecov/codecov-action\r\n* https://github.com/coveralls-clients/coveralls-python\r\n\r\nI'm going to try https://codecov.io/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643660427", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643660427, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY2MDQyNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T18:29:30Z", "updated_at": "2020-06-13T18:29:36Z", "author_association": "OWNER", "body": "This one looks easy enough to fix:\r\n\r\n\"Coverage_for_datasette_actor_auth_cookie_py__84_\"", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643658036", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643658036, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY1ODAzNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T18:08:13Z", "updated_at": "2020-06-13T18:08:13Z", "author_association": "OWNER", "body": "From digging through that report it looks like the majority stuff that isn't fully covered is corner-cases... which are the kind of things I really do want the tests to catch.\r\n\r\nI'm not entirely ready to commit to 100%, but I'm going to start digging through and seeing how close I can get. If I can get to 98% (I'm on 91% already) I may as well push all the way to 100.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643657287", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643657287, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY1NzI4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T18:01:39Z", "updated_at": "2020-06-13T18:01:39Z", "author_association": "OWNER", "body": "Added `--cov-report html` and got this report: https://static.simonwillison.net/static/2020/htmlcov-issue-841/index.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643656053", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643656053, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY1NjA1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T17:50:34Z", "updated_at": "2020-06-13T17:50:34Z", "author_association": "OWNER", "body": "Added a `.coveragerc` file:\r\n```\r\n[run]\r\nomit = datasette/_version.py, datasette/utils/shutil_backport.py\r\n```\r\nAnd ran again: `pytest --cov=datasette --cov-config=.coveragerc`\r\n```\r\nName Stmts Miss Cover\r\n------------------------------------------------------\r\ndatasette/__init__.py 3 0 100%\r\ndatasette/__main__.py 3 3 0%\r\ndatasette/actor_auth_cookie.py 19 3 84%\r\ndatasette/app.py 499 27 95%\r\ndatasette/cli.py 157 45 71%\r\ndatasette/database.py 233 17 93%\r\ndatasette/default_permissions.py 39 0 100%\r\ndatasette/facets.py 209 24 89%\r\ndatasette/filters.py 122 7 94%\r\ndatasette/hookspecs.py 19 0 100%\r\ndatasette/inspect.py 37 23 38%\r\ndatasette/plugins.py 34 6 82%\r\ndatasette/publish/__init__.py 0 0 100%\r\ndatasette/publish/cloudrun.py 55 2 96%\r\ndatasette/publish/common.py 19 1 95%\r\ndatasette/publish/heroku.py 95 13 86%\r\ndatasette/renderer.py 63 4 94%\r\ndatasette/sql_functions.py 4 0 100%\r\ndatasette/tracer.py 85 16 81%\r\ndatasette/utils/__init__.py 503 31 94%\r\ndatasette/utils/asgi.py 253 25 90%\r\ndatasette/version.py 4 0 100%\r\ndatasette/views/__init__.py 0 0 100%\r\ndatasette/views/base.py 288 19 93%\r\ndatasette/views/database.py 120 2 98%\r\ndatasette/views/index.py 57 2 96%\r\ndatasette/views/special.py 72 16 78%\r\ndatasette/views/table.py 418 18 96%\r\n------------------------------------------------------\r\nTOTAL 3410 304 91%\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643655108", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643655108, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY1NTEwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T17:43:15Z", "updated_at": "2020-06-13T17:43:15Z", "author_association": "OWNER", "body": "Using https://pypi.org/project/pytest-cov/ and running `pytest --cov=datasette`:\r\n```\r\n---------- coverage: platform darwin, python 3.7.7-final-0 -----------\r\nName Stmts Miss Cover\r\n--------------------------------------------------------\r\ndatasette/__init__.py 3 0 100%\r\ndatasette/__main__.py 3 3 0%\r\ndatasette/_version.py 277 152 45%\r\ndatasette/actor_auth_cookie.py 19 3 84%\r\ndatasette/app.py 499 27 95%\r\ndatasette/cli.py 157 45 71%\r\ndatasette/database.py 233 17 93%\r\ndatasette/default_permissions.py 39 0 100%\r\ndatasette/facets.py 209 24 89%\r\ndatasette/filters.py 122 7 94%\r\ndatasette/hookspecs.py 19 0 100%\r\ndatasette/inspect.py 37 23 38%\r\ndatasette/plugins.py 34 6 82%\r\ndatasette/publish/__init__.py 0 0 100%\r\ndatasette/publish/cloudrun.py 55 2 96%\r\ndatasette/publish/common.py 19 1 95%\r\ndatasette/publish/heroku.py 95 13 86%\r\ndatasette/renderer.py 63 4 94%\r\ndatasette/sql_functions.py 4 0 100%\r\ndatasette/tracer.py 85 16 81%\r\ndatasette/utils/__init__.py 503 31 94%\r\ndatasette/utils/asgi.py 253 25 90%\r\ndatasette/utils/shutil_backport.py 44 40 9%\r\ndatasette/version.py 4 0 100%\r\ndatasette/views/__init__.py 0 0 100%\r\ndatasette/views/base.py 288 19 93%\r\ndatasette/views/database.py 120 2 98%\r\ndatasette/views/index.py 57 2 96%\r\ndatasette/views/special.py 72 16 78%\r\ndatasette/views/table.py 418 18 96%\r\n--------------------------------------------------------\r\nTOTAL 3731 496 87%\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/834#issuecomment-643648359", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/834", "id": 643648359, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzY0ODM1OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T16:47:29Z", "updated_at": "2020-06-13T16:47:29Z", "author_association": "OWNER", "body": "Implementing this is proving surprisingly tricky, because of the need to be able to optionally `await` the returned value. It's a bit of a fiddle to get this to work within unit tests because they run in non-async functions - due to this cunning `async_to_sync` usage in the test client:\r\n\r\nhttps://github.com/simonw/datasette/blob/b906030235efbdff536405d66078f4868ce0d3bd/tests/fixtures.py#L115-L133\r\n\r\nI could switch to using `async def test_*` functions decorated with `@pytest.mark.asyncio` but I'd rather not re-engineer the entire test suite just for this one feature, so I'll try to find another way.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637342551, "label": "startup() plugin hook"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/841#issuecomment-643576372", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/841", "id": 643576372, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzU3NjM3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-13T06:08:34Z", "updated_at": "2020-06-13T06:08:34Z", "author_association": "OWNER", "body": "Starlette achieves this. https://github.com/encode/starlette", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 638104520, "label": "Research feasibility of 100% test coverage"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/834#issuecomment-643510240", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/834", "id": 643510240, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzUxMDI0MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T22:40:26Z", "updated_at": "2020-06-12T22:40:26Z", "author_association": "OWNER", "body": "Another use-case: plugins that need their own database with the correct tables. They can write to the database on startup to create their tables.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637342551, "label": "startup() plugin hook"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/834#issuecomment-643509358", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/834", "id": 643509358, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzUwOTM1OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T22:36:37Z", "updated_at": "2020-06-12T22:36:37Z", "author_association": "OWNER", "body": "This should be able to optionally return an async function which is then awaited.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637342551, "label": "startup() plugin hook"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/805#issuecomment-643501428", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/805", "id": 643501428, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzUwMTQyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T22:06:08Z", "updated_at": "2020-06-12T22:06:08Z", "author_association": "OWNER", "body": "This needs the `startup` hook, see https://github.com/simonw/datasette/issues/834#issuecomment-643501064", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632724154, "label": "Writable canned queries live demo on Glitch"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/834#issuecomment-643501064", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/834", "id": 643501064, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzUwMTA2NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T22:04:43Z", "updated_at": "2020-06-12T22:04:43Z", "author_association": "OWNER", "body": "Another use-case for this: I want to use the `--root` option on Glitch but it gives me a 127.0.0.1 URL. Glitch has a `PROJECT_DOMAIN` environment variable which tells me the URL. A `datasette-glitch` plugin could use a `startup` hook to output the correct login URL.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637342551, "label": "startup() plugin hook"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/840#issuecomment-643454625", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/840", "id": 643454625, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzQ1NDYyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T19:47:38Z", "updated_at": "2020-06-12T19:47:53Z", "author_association": "OWNER", "body": "Another problem: what to display in the \"you are logged in as\", since we don't dictate an actor design.\r\n\r\nI'm going to use a includes template for this that can easily be over-ridden by administrators or by plugins.\r\n\r\nThe default will look for the first available of the following keys:\r\n\r\n- display\r\n- name\r\n- username\r\n- login\r\n- id", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637966833, "label": "Log out mechanism for clearing ds_actor cookie"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/840#issuecomment-643453128", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/840", "id": 643453128, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzQ1MzEyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T19:43:15Z", "updated_at": "2020-06-12T19:43:15Z", "author_association": "OWNER", "body": "I don't like how this often involves a logout link that can be maliciously activated.\r\n\r\nI'm going to use a CSRF protected form button styled to look like a link instead.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637966833, "label": "Log out mechanism for clearing ds_actor cookie"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643414646", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40", "id": 643414646, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzQxNDY0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T18:06:48Z", "updated_at": "2020-06-12T18:06:48Z", "author_association": "MEMBER", "body": "That fixed it.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637899539, "label": "Demo deploy is broken"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/sqlite-utils/issues/115#issuecomment-643406939", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/115", "id": 643406939, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzQwNjkzOQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T17:51:11Z", "updated_at": "2020-06-12T17:51:11Z", "author_association": "OWNER", "body": "https://github.com/simonw/sqlite-utils/blob/03ee97d2258254581bea72842518904fc1cbe60f/tests/test_cli.py#L1112-L1128", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637889964, "label": "Ability to execute insert/update statements with the CLI"}, "performed_via_github_app": null} {"html_url": "https://github.com/dogsheep/github-to-sqlite/issues/40#issuecomment-643393506", "issue_url": "https://api.github.com/repos/dogsheep/github-to-sqlite/issues/40", "id": 643393506, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzM5MzUwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T17:21:14Z", "updated_at": "2020-06-12T17:21:14Z", "author_association": "MEMBER", "body": "I only install SQLite for this:\r\n\r\nhttps://github.com/dogsheep/github-to-sqlite/blob/c0d54e0260468be38152293df5abd775c068495d/.github/workflows/deploy-demo.yml#L77-L78\r\n\r\nI'm going to remove the need to install sqlite3 by making this possible with sqlite-utils: https://github.com/simonw/sqlite-utils/issues/115", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637899539, "label": "Demo deploy is broken"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/838#issuecomment-643083451", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/838", "id": 643083451, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzA4MzQ1MQ==", "user": {"value": 79913, "label": "tsibley"}, "created_at": "2020-06-12T06:04:14Z", "updated_at": "2020-06-12T06:04:14Z", "author_association": "NONE", "body": "Hmm, I haven't tried removing `ProxyPassReverse`, but it doesn't touch the HTML, which is the issue I'm seeing. You can read the [documentation here](https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypassreverse). `ProxyPassReverse` is a standard directive when proxying with Apache. I've used it dozens of times with other applications.\r\n\r\nLooking a little more at the code, I think the issue here is that the behaviour of `base_url` makes sense when Datasette is _mounted_ at a path within a larger application, but not when HTTP requests are being _proxied_ to it.\r\n\r\nIn a _mount_ situation, it is perfectly fine to construct URLs reusing the domain and path from the request. In a _proxy_ situation, it never is, as the domain and path in the request are not the domain and path that the non-proxy client actually needs to use. That is, links which include the Apache \u2192 Datasette request origin, `localhost:8001`, instead of the browser \u2192 Apache request origin, `example.com`, will be broken.\r\n\r\nThe tests you pointed to also reflect this in two ways:\r\n\r\n1. They strip a leading `http://localhost`, allowing such URLs in the facet links to pass, but inclusion of that in a proxy situation would mean the URL is broken.\r\n\r\n2. The test client emits direct ASGI events instead of actual proxied HTTP requests. The headers of these ASGI events don't reflect the way an HTTP proxy works; instead they pass through the original request path which contains `base_url`. This works because Datasette responds to requests equivalently at either `/\u2026` or `/{base_url}/\u2026`, which makes some sense in a _mount_ situation but is unconventional (albeit workable) for a proxied app.\r\n\r\nApps that support being proxied automatically support being mounted, but apps that only support being mounted don't automatically support being proxied.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637395097, "label": "Incorrect URLs when served behind a proxy with base_url set"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/806#issuecomment-643010591", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/806", "id": 643010591, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzAxMDU5MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T01:13:06Z", "updated_at": "2020-06-12T01:13:06Z", "author_association": "OWNER", "body": "Tests are passing again: https://github.com/simonw/datasette/commit/9ae0d483ead93c0832142e5dc85959ae3c8f73ea", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632753851, "label": "Release Datasette 0.44"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/806#issuecomment-643000948", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/806", "id": 643000948, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MzAwMDk0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T00:34:21Z", "updated_at": "2020-06-12T00:34:21Z", "author_association": "OWNER", "body": "I'm going to add https://github.com/simonw/datasette-auth-tokens and https://github.com/simonw/datasette-permissions-sql to the documentation and release notes in a few places.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632753851, "label": "Release Datasette 0.44"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/806#issuecomment-642998097", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/806", "id": 642998097, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk5ODA5Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T00:26:00Z", "updated_at": "2020-06-12T00:26:00Z", "author_association": "OWNER", "body": "OK, I'm ready to ship. Last check of the release notes, then I'll update the news section in the README and release 0.44!", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632753851, "label": "Release Datasette 0.44"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/838#issuecomment-642993277", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/838", "id": 642993277, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk5MzI3Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T00:18:26Z", "updated_at": "2020-06-12T00:18:50Z", "author_association": "OWNER", "body": "Have you tried this without the `ProxyPassReverse` directive? I'm worried that might be confusing Datasette.\r\n\r\nThis is the test I used to ensure this feature works - it scrapes all of the links on a bunch of different pages. Could it be missing something here?\r\n\r\nhttps://github.com/simonw/datasette/blob/647c5ff0f3e8140f40d7f41f0874ce4e1f4df65c/tests/test_html.py#L1233-L1274\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637395097, "label": "Incorrect URLs when served behind a proxy with base_url set"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/824#issuecomment-642991513", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/824", "id": 642991513, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk5MTUxMw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-12T00:11:50Z", "updated_at": "2020-06-12T00:11:50Z", "author_association": "OWNER", "body": "Done: https://github.com/simonw/datasette-auth-tokens and https://pypi.org/project/datasette-auth-tokens/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 635108074, "label": "Example authentication plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/833#issuecomment-642958225", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/833", "id": 642958225, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk1ODIyNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T22:15:32Z", "updated_at": "2020-06-11T22:15:32Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette/blob/29c5ff493ad7918b8fc44ea7920b41530e56dd5d/tests/test_permissions.py#L327-L348", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637253789, "label": "/-/metadata and so on should respect view-instance permission"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/824#issuecomment-642953605", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/824", "id": 642953605, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk1MzYwNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T22:02:32Z", "updated_at": "2020-06-11T22:02:32Z", "author_association": "OWNER", "body": "`datasette-auth-tokens` can be the name. I can get a simple initial version of it running pretty quickly.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 635108074, "label": "Example authentication plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/824#issuecomment-642952962", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/824", "id": 642952962, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk1Mjk2Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T22:01:58Z", "updated_at": "2020-06-11T22:01:58Z", "author_association": "OWNER", "body": "Alternative idea: a plugin that handles Bearer token authentication. Uses `metadata.json` with secret plugin values to map an incoming token to an actor dictionary, which can then be mapped to permissions.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 635108074, "label": "Example authentication plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/824#issuecomment-642951150", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/824", "id": 642951150, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk1MTE1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T22:00:17Z", "updated_at": "2020-06-11T22:00:17Z", "author_association": "OWNER", "body": "I got this working: https://github.com/simonw/datasette-auth-github/pull/64\r\n\r\nJust one problem: it uses the existing `ds_actor` cookie, which means it doesn't actually exercise the `actor_from_request` plugin!\r\n\r\nIt does use `register_routes` though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 635108074, "label": "Example authentication plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/220#issuecomment-642944645", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/220", "id": 642944645, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjk0NDY0NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T21:49:55Z", "updated_at": "2020-06-11T21:49:55Z", "author_association": "OWNER", "body": "I'm OK with not implementing this - I've got used to the existing mechanism, and it doesn't frustrate me enough to work on this more.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 314847571, "label": "Investigate syntactic sugar for plugins"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/832#issuecomment-642907021", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/832", "id": 642907021, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjkwNzAyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T20:20:35Z", "updated_at": "2020-06-11T20:20:35Z", "author_association": "OWNER", "body": "I think the new `.check_permissions()` should be a documented utility that is available to plugins.\r\n Maybe a method on `datasette`?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636722501, "label": "Having view-table permission but NOT view-database should still grant access to /db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/832#issuecomment-642906681", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/832", "id": 642906681, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjkwNjY4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T20:19:47Z", "updated_at": "2020-06-11T20:20:02Z", "author_association": "OWNER", "body": "So for the following:\r\n```\r\nawait self.check_permissions(request, [\r\n (\"view-table\", (database, table)),\r\n (\"view-database\", database),\r\n \"view-instance\",\r\n])\r\n```\r\nThe logic is: if the first test returns `True`, you get access. If it returns `False` you are denied. If it says `None` then move on to the next check in the list and repeat.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636722501, "label": "Having view-table permission but NOT view-database should still grant access to /db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/833#issuecomment-642905424", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/833", "id": 642905424, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjkwNTQyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T20:16:41Z", "updated_at": "2020-06-11T20:16:41Z", "author_association": "OWNER", "body": "I'll add a new test in `test_permissions.py` which locks down an instance and then loops through paths as the anonymous user making sure they aren't accessible.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637253789, "label": "/-/metadata and so on should respect view-instance permission"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/833#issuecomment-642902208", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/833", "id": 642902208, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjkwMjIwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T20:08:57Z", "updated_at": "2020-06-11T20:08:57Z", "author_association": "OWNER", "body": "I'm tempted to add a `view-instance` check before routing any URLs, but that wouldn't be compatible with the idea in #832 that having `view-table` should be enough to view a table even if you don't pass `view-instance`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637253789, "label": "/-/metadata and so on should respect view-instance permission"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/833#issuecomment-642874724", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/833", "id": 642874724, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjg3NDcyNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T19:07:49Z", "updated_at": "2020-06-11T19:07:49Z", "author_association": "OWNER", "body": "A live demo running the `datasette-auth-github` plugin will help demonstrate this.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 637253789, "label": "/-/metadata and so on should respect view-instance permission"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/801#issuecomment-642870553", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/801", "id": 642870553, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjg3MDU1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T18:58:49Z", "updated_at": "2020-06-11T18:58:49Z", "author_association": "OWNER", "body": "I've implemented this in a plugin instead: https://github.com/simonw/datasette-permissions-sql", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 631932926, "label": "allow_by_query setting for configuring permissions with a SQL statement"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/832#issuecomment-642795966", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/832", "id": 642795966, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjc5NTk2Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T16:37:21Z", "updated_at": "2020-06-11T16:37:21Z", "author_association": "OWNER", "body": "How would I document this? Probably in another section on https://datasette.readthedocs.io/en/latest/authentication.html#permissions\r\n\r\nBut I'd also need to add documentation to the individual views stating what permissions are checked and in what order. I could do that on this page: https://datasette.readthedocs.io/en/latest/pages.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636722501, "label": "Having view-table permission but NOT view-database should still grant access to /db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/809#issuecomment-642772344", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/809", "id": 642772344, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjc3MjM0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T16:01:15Z", "updated_at": "2020-06-11T16:01:15Z", "author_association": "OWNER", "body": "```\r\ndatasette package fixtures.db --secret woot --branch master\r\nSending build context to Docker daemon 260.6kB\r\nStep 1/9 : FROM python:3.8\r\n3.8: Pulling from library/python\r\ne9afc4f90ab0: Downloading [=======> ] 7.195MB/50.39MB\r\n989e6b19a265: Downloading [============================> ] 4.475MB/7.812MB\r\naf14b6c2f878: Downloading [===========================> ] 5.422MB/9.996MB\r\n5573c4b30949: Waiting \r\n11a88e764313: Waiting \r\nee776f0e36af: Waiting \r\n513c90a1afc3: Waiting \r\ndf9b9e95bdb9: Waiting \r\n86c9edb54464: Waiting \r\n...\r\ndatasette package fixtures.db --secret woot --branch master\r\ndocker run -p 8001:8001 a155798bd842\r\n```\r\nThis works too.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632919570, "label": "Publish secrets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/809#issuecomment-642754589", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/809", "id": 642754589, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjc1NDU4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T15:45:25Z", "updated_at": "2020-06-11T15:45:25Z", "author_association": "OWNER", "body": " datasette publish cloudrun fixtures.db --service datasette-publish-secret --branch=master\r\n\r\nhttps://datasette-publish-secret-j7hipcg4aq-uw.a.run.app/-/messages", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632919570, "label": "Publish secrets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/809#issuecomment-642750790", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/809", "id": 642750790, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjc1MDc5MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T15:42:23Z", "updated_at": "2020-06-11T15:42:23Z", "author_association": "OWNER", "body": " datasette publish heroku fixtures.db -n datasette-publish-secret --branch=master\r\n\r\nhttps://datasette-publish-secret.herokuapp.com/-/messages - Heroku works.\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632919570, "label": "Publish secrets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/pull/809#issuecomment-642745518", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/809", "id": 642745518, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjc0NTUxOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T15:38:51Z", "updated_at": "2020-06-11T15:38:51Z", "author_association": "OWNER", "body": "The way to manually test this is to publish a database to each provider and then check that the `/-/messages` debug tool works.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 632919570, "label": "Publish secrets"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/832#issuecomment-642741930", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/832", "id": 642741930, "node_id": "MDEyOklzc3VlQ29tbWVudDY0Mjc0MTkzMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T15:35:53Z", "updated_at": "2020-06-11T15:36:05Z", "author_association": "OWNER", "body": "May the fix here is to implement a `.check_permissions()` method which passes when the first permission passes?\r\n```python\r\nawait self.check_permissions(request, [\r\n (\"view-table\", (database, table)),\r\n (\"view-database\", database),\r\n \"view-instance\",\r\n])\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636722501, "label": "Having view-table permission but NOT view-database should still grant access to /db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/394#issuecomment-642522285", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/394", "id": 642522285, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjUyMjI4NQ==", "user": {"value": 58298410, "label": "LVerneyPEReN"}, "created_at": "2020-06-11T09:15:19Z", "updated_at": "2020-06-11T09:15:19Z", "author_association": "NONE", "body": "Hi @wragge,\r\n\r\nThis looks great, thanks for the share! I refactored it into a self-contained function, binding on a random available TCP port (multi-user context). I am using subprocess API directly since the `%run` magic was leaving defunct process behind :/\r\n\r\n![image](https://user-images.githubusercontent.com/58298410/84367566-b5d0d500-abd4-11ea-96e2-f5c05a28e506.png)\r\n\r\n```python\r\nimport socket\r\n\r\nfrom signal import SIGINT\r\nfrom subprocess import Popen, PIPE\r\n\r\nfrom IPython.display import display, HTML\r\nfrom notebook.notebookapp import list_running_servers\r\n\r\n\r\ndef get_free_tcp_port():\r\n \"\"\"\r\n Get a free TCP port.\r\n \"\"\"\r\n tcp = socket.socket(socket.AF_INET, socket.SOCK_STREAM)\r\n tcp.bind(('', 0))\r\n _, port = tcp.getsockname()\r\n tcp.close()\r\n return port\r\n\r\n\r\ndef datasette(database):\r\n \"\"\"\r\n Run datasette on an SQLite database.\r\n \"\"\"\r\n # Get current running servers\r\n servers = list_running_servers()\r\n\r\n # Get the current base url\r\n base_url = next(servers)['base_url']\r\n\r\n # Get a free port\r\n port = get_free_tcp_port()\r\n\r\n # Create a base url for Datasette suing the proxy path\r\n proxy_url = f'{base_url}proxy/absolute/{port}/'\r\n\r\n # Display a link to Datasette\r\n display(HTML(f'

View Datasette (Click on the stop button to close the Datasette server)

'))\r\n\r\n # Launch Datasette\r\n with Popen(\r\n [\r\n 'python', '-m', 'datasette', '--',\r\n database,\r\n '--port', str(port),\r\n '--config', f'base_url:{proxy_url}'\r\n ],\r\n stdout=PIPE,\r\n stderr=PIPE,\r\n bufsize=1,\r\n universal_newlines=True\r\n ) as p:\r\n print(p.stdout.readline(), end='')\r\n while True:\r\n try:\r\n line = p.stderr.readline()\r\n if not line:\r\n break\r\n print(line, end='')\r\n exit_code = p.poll()\r\n except KeyboardInterrupt:\r\n p.send_signal(SIGINT)\r\n```\r\n\r\nIdeally, I'd like some extra magic to notify users when they are leaving the closing the notebook tab and make them terminate the running datasette processes. I'll be looking for it.", "reactions": "{\"total_count\": 1, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 1, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 396212021, "label": "base_url configuration setting"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/818#issuecomment-642420375", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/818", "id": 642420375, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjQyMDM3NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T05:40:07Z", "updated_at": "2020-06-11T05:40:07Z", "author_association": "OWNER", "body": "https://github.com/simonw/datasette-permissions-sql is now released as a 0.1a here: https://pypi.org/project/datasette-permissions-sql/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 634917088, "label": "Example permissions plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/832#issuecomment-642412017", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/832", "id": 642412017, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjQxMjAxNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-11T05:13:59Z", "updated_at": "2020-06-11T05:13:59Z", "author_association": "OWNER", "body": "Relevant code:\r\n\r\nhttps://github.com/simonw/datasette/blob/ce4958018ede00fbdadf0c37a99889b6901bfb9b/datasette/views/table.py#L267-L272", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636722501, "label": "Having view-table permission but NOT view-database should still grant access to /db/table"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/831#issuecomment-642324847", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/831", "id": 642324847, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjMyNDg0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T23:50:55Z", "updated_at": "2020-06-10T23:50:55Z", "author_association": "OWNER", "body": "Actually I'm not sure about this. If `\"allow\": null` means \"no-one can do this\", what's the allow block syntax for \"everyone can do this\"?\r\n\r\nIt could be `\"allow\": {}` - but that's not intuitive because normally the allow block shows keys that need to match. `{}` suggests to me that no matches are possible.\r\n\r\nSo I think I'm going to stick with the current mechanism, which is that `\"allow\": null` means \"anyone can do this\" and `\"allow\": {}` means \"no-one can do this\".", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636614868, "label": "It would be more intuitive if \"allow\": none meant \"no-one can do this\""}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/818#issuecomment-642231871", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/818", "id": 642231871, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjIzMTg3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T20:11:50Z", "updated_at": "2020-06-10T20:11:50Z", "author_association": "OWNER", "body": "`datasette-permissions-sql`\r\n```yaml\r\nplugins:\r\n datasette-permissions-sql:\r\n view-instance: |-\r\n select count(*) from users where admin = 1 and id = :id\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 634917088, "label": "Example permissions plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/818#issuecomment-642230499", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/818", "id": 642230499, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjIzMDQ5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T20:08:46Z", "updated_at": "2020-06-10T20:09:26Z", "author_association": "OWNER", "body": "What's a simple but useful plugin I could release that exercises this hook?\r\n\r\nIdeally one which executes permission checks against the database somehow.\r\n\r\nI could do a simplest-possible implementation of the idea in #801 (allow-by-query).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 634917088, "label": "Example permissions plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/818#issuecomment-642229899", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/818", "id": 642229899, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjIyOTg5OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T20:07:36Z", "updated_at": "2020-06-10T20:07:36Z", "author_association": "OWNER", "body": "New policy in 9f236c4 dictates that this should be in Milestone 0.44 after all:\r\n\r\n> * **New plugin hooks** should only be shipped if accompanied by a separate release of a non-demo plugin that uses them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 634917088, "label": "Example permissions plugin"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/829#issuecomment-642217520", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/829", "id": 642217520, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjIxNzUyMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T19:41:35Z", "updated_at": "2020-06-10T19:41:35Z", "author_association": "OWNER", "body": "I didn't bother with the alternative epoch - it only shaves off two or three bytes from the cookie.\r\n\r\nDocumentation for the new `ds_actor` cookie shape is here: https://datasette.readthedocs.io/en/latest/authentication.html#the-ds-actor-cookie", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636426530, "label": "Ability to set ds_actor cookie such that it expires"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/829#issuecomment-642178604", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/829", "id": 642178604, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjE3ODYwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T18:18:36Z", "updated_at": "2020-06-10T18:20:19Z", "author_association": "OWNER", "body": "Even shorter: encode an integer that is the difference between that expiry timestamp and a more recent epoch - June 1st 2020 will do.\r\n\r\n```\r\n>>> import datetime, calendar\r\n>>> calendar.timegm(datetime.date(2020, 6, 1).timetuple())\r\n1590969600\r\n>>> import baseconv\r\n>>> baseconv.base62.encode(int(time.time() - 1590969600))\r\n'3XST'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636426530, "label": "Ability to set ds_actor cookie such that it expires"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/829#issuecomment-642176180", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/829", "id": 642176180, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjE3NjE4MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T18:14:02Z", "updated_at": "2020-06-10T18:14:15Z", "author_association": "OWNER", "body": "And the `e` key can be `null`or missing for \"never expires\".", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636426530, "label": "Ability to set ds_actor cookie such that it expires"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/829#issuecomment-642175892", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/829", "id": 642175892, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjE3NTg5Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T18:13:26Z", "updated_at": "2020-06-10T18:13:26Z", "author_association": "OWNER", "body": "I'm going with `expires_at` - except to keep the cookies shorter the key will be called `e` and the actor will go in `a`, like this:\r\n\r\n```json\r\n{\r\n \"e\": \"1UuHoo\",\r\n \"a\": {\"id\": \"root\"}\r\n}\r\n```\r\nThat `e` value is a base64 encoded expiry integer timestamp (again for a shorter cookie) - using https://pypi.org/project/python-baseconv/", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636426530, "label": "Ability to set ds_actor cookie such that it expires"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/829#issuecomment-642174272", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/829", "id": 642174272, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjE3NDI3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T18:10:13Z", "updated_at": "2020-06-10T18:10:13Z", "author_association": "OWNER", "body": "Some options:\r\n\r\n- Redesign the `ds_actor` cookie to be `{\"expires_at\": 1591811250, \"actor\": ...}` - check if it has expired in that default `actor_from_request` hook\r\n- Let plugins set an additional cookie of some sort\r\n- Expect plugins that care about this to set a cookie with a different name and implement their own `actor_from_request` against that", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636426530, "label": "Ability to set ds_actor cookie such that it expires"}, "performed_via_github_app": null} {"html_url": "https://github.com/simonw/datasette/issues/829#issuecomment-642161210", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/829", "id": 642161210, "node_id": "MDEyOklzc3VlQ29tbWVudDY0MjE2MTIxMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-06-10T17:45:58Z", "updated_at": "2020-06-10T17:45:58Z", "author_association": "OWNER", "body": "`itsdangerous` has this ability but you specify the max-age when you call unsign: https://itsdangerous.palletsprojects.com/en/1.1.x/timed/\r\n\r\n> s.unsign(string, max_age=5)\r\n> Traceback (most recent call last):\r\n> ...\r\n> itsdangerous.exc.SignatureExpired: Signature age 15 > 5 seconds\r\n\r\nI currently only decode the `ds_actor` cookie in one place: https://github.com/simonw/datasette/blob/d828abaddec0dce3ec4b4eeddc3a74384e52cf34/datasette/actor_auth_cookie.py#L5-L12\r\n\r\nIf plugins want to be able to set their own policies on how long the `ds_actor` cookie should remain valid, how do I know to listen to them when decoding the cookie here?", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 636426530, "label": "Ability to set ds_actor cookie such that it expires"}, "performed_via_github_app": null}