home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

34 rows where issue = 1058896236 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 4

  • simonw 31
  • mrchrisadams 1
  • steren 1
  • glasnt 1

author_association 3

  • OWNER 31
  • NONE 2
  • CONTRIBUTOR 1

issue 1

  • Deploy a live instance of demos/apache-proxy · 34 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
976117989 https://github.com/simonw/datasette/issues/1522#issuecomment-976117989 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46LmDl glasnt 813732 2021-11-23T03:00:34Z 2021-11-23T03:00:34Z CONTRIBUTOR

I tried deploying the most recent version of the Dockerfile in this thread (link to comment), and after trying a few different different combinations, I was only successful when I used --no-cpu-throttling ("CPU Is always allocated" in the UI)

Using this method, I got a very similar issue to you: The first time I'd load the site I'd get a 503. But after that first load, I didn't get the issue again. It would re-occur if the service started from cold boot.

I suspect this is a race condition in the supervisord configuration. The errors I got were the same Connection refused: AH00957: http: attempt to connect to 127.0.0.1:8001 (127.0.0.1) failed, and that seems to indicate that datasette hadn't yet started.

Looking at the order of logs getting back, the processes reported successfully completing loading after the first 503 was returned, so that makes me think race condition.

I can replicate this locally, if I docker run and request localhost:5000/prefix before I get the datasette entered RUNNING state message. Cloud Run wakes up when requests are received, so this test would semi-replicate that, but local docker would be the equivalent of a persistent process, hence it doesn't normally exhibit the same issues.

Unfortunately supervisor/supervisor issue 122 (not linking as to prevent cross-project link spam) seems to say that dependency chaining is a feature that's been asked for for a long time, but hasn't been implemented. You could try some suggestions in that thread.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
976023405 https://github.com/simonw/datasette/issues/1522#issuecomment-976023405 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46LO9t steren 360895 2021-11-23T00:08:07Z 2021-11-23T00:08:07Z NONE

If you suspect that Cloud Run throttled CPU could be the cause, you can request to have CPU always allocated with gcloud beta run deploy --no-cpu-throttling (read more)

It could also be the Cloud Run sandbox that somehow gets in the way here, in which case I recommend testing with the second generation execution environment: gcloud beta run deploy --execution-environment gen2

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974695111 https://github.com/simonw/datasette/issues/1522#issuecomment-974695111 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46GKrH simonw 9599 2021-11-20T18:52:11Z 2021-11-20T18:52:11Z OWNER

The demo is now live on https://datasette-apache-proxy-demo.fly.dev/prefix/

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974693350 https://github.com/simonw/datasette/issues/1522#issuecomment-974693350 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46GKPm simonw 9599 2021-11-20T18:39:27Z 2021-11-20T18:39:27Z OWNER

I'm going to go with Fly instead for this, especially as I can keep it within their free tier (and iI want to get more familiar with their platform).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974692546 https://github.com/simonw/datasette/issues/1522#issuecomment-974692546 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46GKDC simonw 9599 2021-11-20T18:34:13Z 2021-11-20T18:34:13Z OWNER

Here's why it failed - the fly.toml file that was generated when I ran flyctl launch had this section: toml [[services]] http_checks = [] internal_port = 8080 processes = ["app"] protocol = "tcp" script_checks = [] But I need internal_port to be 80 for Apache, so I changed that and ran flyctl deploy --build-arg DATASETTE_REF=main again - and it worked!

https://floral-dust-4577.fly.dev/prefix/ - not seeing any 503 errors there.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974685095 https://github.com/simonw/datasette/issues/1522#issuecomment-974685095 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46GIOn simonw 9599 2021-11-20T17:42:25Z 2021-11-20T17:42:25Z OWNER

I tried to deploy it to Fly - initially using flyctl launch but then switching to flyctl deploy so I could use the --build-arg option (posted a feature request here).

Almost got it working, but it failed the health check: `` % cd datasette/demos/apache-proxy apache-proxy % flyctl launch Creating app in /Users/simon/Dropbox/Development/datasette/demos/apache-proxy Scanning source code Detected Dockerfile app Automatically selected personal organization: Simon Willison ? Select region: sjc (Sunnyvale, California (US)) Created app floral-dust-4577 in organization personal Wrote config file fly.toml Your app is ready. Deploy withflyctl deploy` ? Would you like to deploy now? Yes Deploying floral-dust-4577 ==> Validating app configuration --> Validating app configuration done Services TCP 80/443 ⇢ 8080 ==> Creating build context --> Creating build context done ==> Building image with Docker Sending build context to Docker daemon 8.704kB ... Error error building: executor failed running [/bin/sh -c pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip]: exit code: 1

I didn't pass the build argument, trying again with flyctl deploy

apache-proxy % flyctl deploy --build-arg DATASETTE_REF=main Update available 0.0.229 -> v0.0.255 Run "flyctl version update" to upgrade Deploying floral-dust-4577 ==> Validating app configuration --> Validating app configuration done Services TCP 80/443 ⇢ 8080 ==> Creating build context --> Creating build context done ==> Building image with Docker Sending build context to Docker daemon 8.704kB [+] Building 15.7s (27/27) ... 0.0s ==> Pushing image to fly The push refers to repository [registry.fly.io/floral-dust-4577] 9bf88c92aa2a: Pushed 3d61728b8391: Pushed ... --> Pushing image done Image: registry.fly.io/floral-dust-4577:deployment-1637429501 Image size: 276 MB ==> Creating release Release v2 created

You can detach the terminal anytime without stopping the deployment Monitoring Deployment

1 desired, 1 placed, 0 healthy, 0 unhealthy [health checks: 1 total, 1 critical]

1 desired, 1 placed, 0 healthy, 1 unhealthy [health checks: 1 total, 1 critical] v0 failed - Failed due to unhealthy allocations - no stable job version to auto revert to Failed Instances

==> Failure #1

Instance ID = 36adac86
Version = 0
Region = sjc
Desired = run
Status = running
Health Checks = 1 total, 1 critical
Restarts = 0
Created = 4m52s ago

Recent Events TIMESTAMP TYPE MESSAGE
2021-11-20T17:32:52Z Received Task received by client 2021-11-20T17:32:52Z Task Setup Building Task Directory 2021-11-20T17:33:02Z Started Task started by client

Recent Logs 2021-11-20T17:32:56Z [info] Unpacking image 2021-11-20T17:33:01Z [info] Preparing kernel init 2021-11-20T17:33:01Z [info] Configuring firecracker 2021-11-20T17:33:02Z [info] Starting virtual machine 2021-11-20T17:33:02Z [info] Starting init (commit: 7943db6)... 2021-11-20T17:33:02Z [info] Preparing to run: /usr/bin/supervisord -c /app/supervisord.conf as root 2021-11-20T17:33:02Z [info] 2021/11/20 17:33:02 listening on [fdaa:0:4ef:a7b:2295:36ad:ac86:2]:22 (DNS: [fdaa::3]:53) 2021-11-20T17:33:02Z [info] 2021-11-20 17:33:02,374 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. 2021-11-20T17:33:02Z [info] 2021-11-20 17:33:02,376 INFO supervisord started with pid 510 2021-11-20T17:33:03Z [info] 2021-11-20 17:33:03,379 INFO spawned: 'apache2' with pid 515 2021-11-20T17:33:03Z [info] 2021-11-20 17:33:03,381 INFO spawned: 'datasette' with pid 516 2021-11-20T17:33:05Z [info] 2021-11-20 17:33:05,068 INFO success: apache2 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2021-11-20T17:33:05Z [info] 2021-11-20 17:33:05,068 INFO success: datasette entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2021-11-20T17:33:28Z [error] Health check status changed 'warning' => 'critical' ***v0 failed - Failed due to unhealthy allocations - no stable job version to auto revert to and deploying as v1 ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974683220 https://github.com/simonw/datasette/issues/1522#issuecomment-974683220 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46GHxU simonw 9599 2021-11-20T17:29:12Z 2021-11-20T17:29:12Z OWNER

As a a sanity check, would it be worth looking at trying to push the multi-process container on another provider of a knative / cloud run / tekton ? I have a somewhat similar use case for a future proejct, so i'm been very grateful to you sharing all the progress in this issue.

That's a great idea. I'll try running on a non-Knative host too (probably Fly - though they actually run containers using Firecracker which ends up being completely different).

Cloud Run are the only Knative host I've used, know of any others aside from Scaleway? They look like they're worth getting familiar with.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974682507 https://github.com/simonw/datasette/issues/1522#issuecomment-974682507 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46GHmL simonw 9599 2021-11-20T17:24:13Z 2021-11-20T17:24:13Z OWNER

I'm going to leave this issue open, tag it as "help wanted" and cross my fingers that someone with Cloud Run deep expertise takes an interest in figuring out what's going wrong here!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974607456 https://github.com/simonw/datasette/issues/1522#issuecomment-974607456 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46F1Rg mrchrisadams 17906 2021-11-20T07:10:11Z 2021-11-20T07:10:11Z NONE

As a a sanity check, would it be worth looking at trying to push the multi-process container on another provider of a knative / cloud run / tekton ? I have a somewhat similar use case for a future proejct, so i'm been very grateful to you sharing all the progress in this issue.

As I understand it, Scaleway also offer a very similar offering using what appear to be many similar components that might at least see if it's an issue with more than one knative based FaaS provider

https://www.scaleway.com/en/serverless-containers/ https://developers.scaleway.com/en/products/containers/api/#main-features

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974605529 https://github.com/simonw/datasette/issues/1522#issuecomment-974605529 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46F0zZ simonw 9599 2021-11-20T06:52:21Z 2021-11-20T06:52:21Z OWNER

I've now tried both Debian and Alpine, and I've tried both tini and supervisord. Each time I get the same result - I get 503 errors for the first dozen or so refreshes of /prefix/ followed by it intermittently working. Absolutely stumped.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974605128 https://github.com/simonw/datasette/issues/1522#issuecomment-974605128 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46F0tI simonw 9599 2021-11-20T06:47:59Z 2021-11-20T06:47:59Z OWNER

I managed to port the whole thing over to Debian - which took a lot of work because their packaged Apache 2 works very differently from the Alpine one.

Once again... I got it working fine on my laptop, but the image deployed to Cloud Run throws 503 errors! ```dockerfile FROM python:3.9.7-slim-bullseye

RUN apt-get update && \ apt-get install -y apache2 supervisor && \ apt clean && \ rm -rf /var/lib/apt && \ rm -rf /var/lib/dpkg/info/*

Apache environment, copied from

https://github.com/ijklim/laravel-benfords-law-app/blob/e9bf385dcaddb62ea466a7b245ab6e4ef708c313/docker/os/Dockerfile

ENV APACHE_DOCUMENT_ROOT=/var/www/html/public ENV APACHE_RUN_USER www-data ENV APACHE_RUN_GROUP www-data ENV APACHE_PID_FILE /var/run/apache2.pid ENV APACHE_RUN_DIR /var/run/apache2 ENV APACHE_LOCK_DIR /var/lock/apache2 ENV APACHE_LOG_DIR /var/log RUN ln -sf /dev/stdout /var/log/apache2-access.log RUN ln -sf /dev/stderr /var/log/apache2-error.log RUN mkdir -p $APACHE_RUN_DIR $APACHE_LOCK_DIR

RUN a2enmod proxy RUN a2enmod proxy_http RUN a2enmod headers

ARG DATASETTE_REF

RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip

Append this to the end of the default httpd.conf file

RUN echo '\n\ <Directory /app/html/>\n\ Options Indexes FollowSymLinks\n\ AllowOverride None\n\ Require all granted\n\ </Directory>\n\ \n\ <VirtualHost *:80>\n\ ServerName localhost\n\ DocumentRoot /app/html\n\ ProxyPreserveHost On\n\ ProxyPass /prefix/ http://127.0.0.1:8001/\n\ Header add X-Proxied-By "Apache2"\n\ </VirtualHost>\n\ ' > /etc/apache2/sites-enabled/000-default.conf

WORKDIR /app RUN mkdir -p /app/html RUN echo 'Datasette' > /app/html/index.html

ADD https://latest.datasette.io/fixtures.db /app/fixtures.db

EXPOSE 80

RUN echo "[supervisord]" >> /app/supervisord.conf RUN echo "nodaemon=true" >> /app/supervisord.conf RUN echo "" >> /app/supervisord.conf RUN echo "[program:apache2]" >> /app/supervisord.conf RUN echo "command=apache2 -D FOREGROUND" >> /app/supervisord.conf RUN echo "" >> /app/supervisord.conf RUN echo "[program:datasette]" >> /app/supervisord.conf RUN echo "command=datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001" >> /app/supervisord.conf

CMD ["/usr/bin/supervisord", "-c", "/app/supervisord.conf"] ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974602459 https://github.com/simonw/datasette/issues/1522#issuecomment-974602459 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46F0Db simonw 9599 2021-11-20T06:15:58Z 2021-11-20T06:15:58Z OWNER

First I'm going to try using Debian Buster as the base image instead of Alpine.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974585374 https://github.com/simonw/datasette/issues/1522#issuecomment-974585374 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Fv4e simonw 9599 2021-11-20T03:28:58Z 2021-11-20T03:28:58Z OWNER

Based on https://medium.com/google-cloud/init-process-for-containers-d03a471fa0cc I might try s6.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974578141 https://github.com/simonw/datasette/issues/1522#issuecomment-974578141 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FuHd simonw 9599 2021-11-20T02:27:23Z 2021-11-20T02:27:23Z OWNER

Aha! This could be the clue I was looking for: https://www.reddit.com/r/googlecloud/comments/fmkx63/comment/fl5csty/?utm_source=reddit&utm_medium=web2x&context=3

Are you processing on a background thread in your container? If so, it's likely your problem, because cloud run will put your app into a low power state between http requests. For long running tasks in cloud run, you need to keep the http connection open, and not return until you are done.

Maybe the datasette & process is being affected by that in some way?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974577949 https://github.com/simonw/datasette/issues/1522#issuecomment-974577949 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FuEd simonw 9599 2021-11-20T02:26:09Z 2021-11-20T02:26:17Z OWNER

So frustrating, that's giving me the same problem after being deployed! 503 errors for the first while, then it starts working.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974577565 https://github.com/simonw/datasette/issues/1522#issuecomment-974577565 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Ft-d simonw 9599 2021-11-20T02:23:07Z 2021-11-20T02:23:07Z OWNER

OK, that works on my laptop - and Ctrl+C quits it, which is nice: apache-proxy % docker run -p 5000:80 --rm datasette-build-arg-demo 2021-11-20 02:22:13,925 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. 2021-11-20 02:22:13,927 INFO supervisord started with pid 1 2021-11-20 02:22:14,931 INFO spawned: 'datasette' with pid 7 2021-11-20 02:22:14,934 INFO spawned: 'httpd' with pid 8 2021-11-20 02:22:16,484 INFO success: datasette entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2021-11-20 02:22:16,484 INFO success: httpd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) ^C 2021-11-20 02:22:26,285 WARN received SIGINT indicating exit request 2021-11-20 02:22:26,286 INFO waiting for datasette, httpd to die 2021-11-20 02:22:26,315 INFO stopped: httpd (exit status 0) 2021-11-20 02:22:26,540 INFO stopped: datasette (exit status 0) Here's my new Dockerfile: ```dockerfile FROM python:3-alpine

RUN apk add --no-cache \ apache2 \ apache2-proxy \ supervisor \ bash

ARG DATASETTE_REF

RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip

Append this to the end of the default httpd.conf file

RUN echo -e 'ServerName localhost\n\ \n\ <Proxy *>\n\ Order deny,allow\n\ Allow from all\n\ </Proxy>\n\ \n\ ProxyPreserveHost On\n\ ProxyPass /prefix/ http://127.0.0.1:8001/\n\ Header add X-Proxied-By "Apache2"' >> /etc/apache2/httpd.conf

RUN echo 'Datasette' > /var/www/localhost/htdocs/index.html

WORKDIR /app

ADD https://latest.datasette.io/fixtures.db /app/fixtures.db

EXPOSE 80

RUN echo "[supervisord]" >> /app/supervisord.conf RUN echo "nodaemon=true" >> /app/supervisord.conf RUN echo "" >> /app/supervisord.conf RUN echo "[program:httpd]" >> /app/supervisord.conf RUN echo "command=httpd -D FOREGROUND" >> /app/supervisord.conf RUN echo "" >> /app/supervisord.conf RUN echo "[program:datasette]" >> /app/supervisord.conf RUN echo "command=datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001" >> /app/supervisord.conf

CMD ["/usr/bin/supervisord", "-c", "/app/supervisord.conf"] ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974577082 https://github.com/simonw/datasette/issues/1522#issuecomment-974577082 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Ft26 simonw 9599 2021-11-20T02:19:27Z 2021-11-20T02:19:27Z OWNER

https://docs.docker.com/config/containers/multi-service_container/ suggests supervisord as a last resort.

https://stackoverflow.com/a/49100302/6083 has a neat looking recipe for than in Alpine:

1. Dockerfile is:

FROM alpine:latest
RUN apk update && apk add --no-cache supervisor openssh nginx
COPY supervisord.conf /etc/supervisord.conf
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisord.conf"]

2. supervisord.conf is:

[supervisord]
nodaemon=true

[program:sshd]
command=/usr/sbin/sshd -D

[program:nginx]
command=nginx -c /etc/nginx/nginx.conf
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974576624 https://github.com/simonw/datasette/issues/1522#issuecomment-974576624 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Ftvw simonw 9599 2021-11-20T02:16:12Z 2021-11-20T02:16:12Z OWNER

Again, that approach worked on my laptop but when deployed to Cloud Run mostly gave me 503 errors for the /prefix/ page, with the occasional 200.

I did this:

```Dockerfile RUN echo "#!/bin/bash" >> start.sh

Start Datasette running in background with &

RUN echo "datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001 &" >> /app/start.sh RUN echo "httpd -D FOREGROUND" >> /app/start.sh

RUN chmod +x /app/start.sh

CMD /app/start.sh ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974576436 https://github.com/simonw/datasette/issues/1522#issuecomment-974576436 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Fts0 simonw 9599 2021-11-20T02:14:45Z 2021-11-20T02:14:45Z OWNER

I'm going to try running Apache with httpd -D FOREGROUND while running datasette &.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974575512 https://github.com/simonw/datasette/issues/1522#issuecomment-974575512 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FteY simonw 9599 2021-11-20T02:09:20Z 2021-11-20T02:09:20Z OWNER

Waiting for health check to begin makes it sound like the container didn't start properly.

That eventually failed, but I did get these in the build logs:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974573616 https://github.com/simonw/datasette/issues/1522#issuecomment-974573616 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FtAw simonw 9599 2021-11-20T01:58:44Z 2021-11-20T01:58:44Z OWNER

Deploy to Cloud Run appears to hang here: Deploying container to Cloud Run service [datasette-apache-proxy-demo] in project [datasette-222320] region [us-central1] ⠧ Deploying... Revision deployment finished. Waiting for health check to begin. ⠧ Creating Revision... . Routing traffic... ✓ Setting IAM Policy... Waiting for health check to begin makes it sound like the container didn't start properly.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974572505 https://github.com/simonw/datasette/issues/1522#issuecomment-974572505 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FsvZ simonw 9599 2021-11-20T01:50:48Z 2021-11-20T01:51:41Z OWNER

I figured out a recipe to run httpd as a service inside Alpine - works great on my laptop, here's my new Dockerfile: ```dockerfile FROM python:3-alpine

openrc gives us rc-service

RUN apk add --no-cache \ openrc \ apache2 \ apache2-proxy \ bash

ARG DATASETTE_REF

RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip

Append this to the end of the default httpd.conf file

RUN echo -e 'ServerName localhost\n\ \n\ <Proxy *>\n\ Order deny,allow\n\ Allow from all\n\ </Proxy>\n\ \n\ ProxyPreserveHost On\n\ ProxyPass /prefix/ http://127.0.0.1:8001/\n\ Header add X-Proxied-By "Apache2"' >> /etc/apache2/httpd.conf

RUN echo 'Datasette' > /var/www/localhost/htdocs/index.html

WORKDIR /app

ADD https://latest.datasette.io/fixtures.db /app/fixtures.db

EXPOSE 80

RUN echo -e "#!/bin/bash\nopenrc default\nrc-service apache2 start;\ndatasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001" > /app/start.sh

RUN echo "#!/bin/bash" >> start.sh RUN echo "openrc default" >> start.sh RUN echo "rc-service apache2 start" >> start.sh RUN echo "datasette /app/fixtures.db --setting base_url '/prefix/' --version-note '${DATASETTE_REF}' -h 0.0.0.0 -p 8001" >> /app/start.sh

RUN chmod +x /app/start.sh

CMD /app/start.sh ``` I'm going to try this on Cloud Run and see if it fixes the 503s

One annoying thing about this: Ctrl+C on my laptop no longer stops the container, I have to docker ps and then docker kill xxx instead.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974565816 https://github.com/simonw/datasette/issues/1522#issuecomment-974565816 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FrG4 simonw 9599 2021-11-20T01:13:39Z 2021-11-20T01:13:39Z OWNER

I have a hunch that running httpd -D FOREGROUND doesn't show error logs, which would explain why I can't use the Cloud Run logs to figure out the reason for the 503s.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974565392 https://github.com/simonw/datasette/issues/1522#issuecomment-974565392 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FrAQ simonw 9599 2021-11-20T01:11:20Z 2021-11-20T01:11:20Z OWNER

Yup, that fixed it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974564712 https://github.com/simonw/datasette/issues/1522#issuecomment-974564712 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Fq1o simonw 9599 2021-11-20T01:07:49Z 2021-11-20T01:10:48Z OWNER

https://apache-proxy-demo.datasette.io/prefix/fixtures/compound_three_primary_keys has broken suggested facet links - they go to https://localhost:8001/prefix/fixtures/compound_three_primary_keys?_facet=pk1#facet-pk1 - but I think that's because I'm missing the ProxyPreserveHost On setting.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974558076 https://github.com/simonw/datasette/issues/1522#issuecomment-974558076 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FpN8 simonw 9599 2021-11-20T00:36:56Z 2021-11-20T00:36:56Z OWNER

That 503 error is really frustrating: I have a deploy running at https://apache-proxy-demo.datasette.io/prefix/ and after a fresh deploy it serves 503 errors for quite a while - then eventually starts working.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974557766 https://github.com/simonw/datasette/issues/1522#issuecomment-974557766 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FpJG simonw 9599 2021-11-20T00:35:25Z 2021-11-20T00:35:25Z OWNER

Wrote a TIL about --build-arg and Cloud Run: https://til.simonwillison.net/cloudrun/using-build-args-with-cloud-run

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974542348 https://github.com/simonw/datasette/issues/1522#issuecomment-974542348 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FlYM simonw 9599 2021-11-19T23:41:47Z 2021-11-19T23:44:07Z OWNER

Do I have to use cloudbuild.yml to specify these? https://stackoverflow.com/a/58327340/6083 and https://stackoverflow.com/a/66232670/6083 suggest I do.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974541971 https://github.com/simonw/datasette/issues/1522#issuecomment-974541971 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FlST simonw 9599 2021-11-19T23:40:32Z 2021-11-19T23:40:32Z OWNER

I want to be able to use build arguments to specify which commit version or branch of Datasette to deploy.

This is proving hard to work out. I have this in my Dockerfile now:

``` ARG DATASETTE_REF

RUN pip install https://github.com/simonw/datasette/archive/${DATASETTE_REF}.zip ``` Which works locally:

docker build -t datasette-apache-proxy-demo . \
  --build-arg DATASETTE_REF=c617e1769ea27e045b0f2907ef49a9a1244e577d

But I can't figure out the right incantation to pass to gcloud build submit.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974523569 https://github.com/simonw/datasette/issues/1522#issuecomment-974523569 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Fgyx simonw 9599 2021-11-19T22:51:10Z 2021-11-19T22:51:10Z OWNER

I wan a GitHub Action which I can manually activate to deploy a new version of that demo... and I want it to bake in the latest release of Datasette so I can use it to demonstrate bug fixes.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974523297 https://github.com/simonw/datasette/issues/1522#issuecomment-974523297 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Fguh simonw 9599 2021-11-19T22:50:31Z 2021-11-19T22:50:31Z OWNER

Demo code is now at: https://github.com/simonw/datasette/tree/main/demos/apache-proxy

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974521687 https://github.com/simonw/datasette/issues/1522#issuecomment-974521687 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FgVX simonw 9599 2021-11-19T22:46:26Z 2021-11-19T22:46:26Z OWNER

Oh weird, it started working: https://datasette-apache-proxy-demo-j7hipcg4aq-uc.a.run.app/prefix/fixtures/sortable

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974506401 https://github.com/simonw/datasette/issues/1522#issuecomment-974506401 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46Fcmh simonw 9599 2021-11-19T22:11:51Z 2021-11-19T22:11:51Z OWNER

This is frustrating: I have the following Dockerfile: ```dockerfile FROM python:3-alpine

RUN apk add --no-cache \ apache2 \ apache2-proxy \ bash

RUN pip install datasette

ENV TINI_VERSION v0.18.0 ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-static /tini RUN chmod +x /tini

Append this to the end of the default httpd.conf file

RUN echo $'ServerName localhost\n\ \n\ <Proxy *>\n\ Order deny,allow\n\ Allow from all\n\ </Proxy>\n\ \n\ ProxyPass /prefix/ http://localhost:8001/\n\ Header add X-Proxied-By "Apache2"' >> /etc/apache2/httpd.conf

RUN echo $'Datasette' > /var/www/localhost/htdocs/index.html

WORKDIR /app

ADD https://latest.datasette.io/fixtures.db /app/fixtures.db

RUN echo $'#!/usr/bin/env bash\n\ set -e\n\ \n\ httpd -D FOREGROUND &\n\ datasette fixtures.db --setting base_url "/prefix/" -h 0.0.0.0 -p 8001 &\n\ \n\ wait -n' > /app/start.sh

RUN chmod +x /app/start.sh

EXPOSE 80 ENTRYPOINT ["/tini", "--", "/app/start.sh"] It works fine when I run it locally: docker build -t datasette-apache-proxy-demo . docker run -p 5000:80 datasette-apache-proxy-demo But when I deploy it to Cloud Run with the following script:bash

!/bin/bash

https://til.simonwillison.net/cloudrun/ship-dockerfile-to-cloud-run

NAME="datasette-apache-proxy-demo" PROJECT=$(gcloud config get-value project) IMAGE="gcr.io/$PROJECT/$NAME"

gcloud builds submit --tag $IMAGE gcloud run deploy \ --allow-unauthenticated \ --platform=managed \ --image $IMAGE $NAME \ --port 80 `` It serves the/page successfully, but hits to/prefix/` return the following 503 error:

Service Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Apache/2.4.51 (Unix) Server at datasette-apache-proxy-demo-j7hipcg4aq-uc.a.run.app Port 80

Cloud Run logs:

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  
974435661 https://github.com/simonw/datasette/issues/1522#issuecomment-974435661 https://api.github.com/repos/simonw/datasette/issues/1522 IC_kwDOBm6k_c46FLVN simonw 9599 2021-11-19T20:33:42Z 2021-11-19T20:33:42Z OWNER

Should just be a case of deploying this Dockerfile:

```Dockerfile FROM python:3-alpine

RUN apk add --no-cache \ apache2 \ apache2-proxy \ bash

RUN pip install datasette

ENV TINI_VERSION v0.18.0 ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-static /tini RUN chmod +x /tini

Append this to the end of the default httpd.conf file

RUN echo $'ServerName localhost\n\ \n\ <Proxy *>\n\ Order deny,allow\n\ Allow from all\n\ </Proxy>\n\ \n\ ProxyPass /foo/bar/ http://localhost:9000/\n\ Header add X-Proxied-By "Apache2"' >> /etc/apache2/httpd.conf

RUN echo $'Datasette' > /var/www/localhost/htdocs/index.html

WORKDIR /app

ADD https://latest.datasette.io/fixtures.db /app/fixtures.db

RUN echo $'#!/usr/bin/env bash\n\ set -e\n\ \n\ httpd -D FOREGROUND &\n\ datasette fixtures.db --setting base_url "/foo/bar/" -p 9000 &\n\ \n\ wait -n' > /app/start.sh

RUN chmod +x /app/start.sh

EXPOSE 80 ENTRYPOINT ["/tini", "--", "/app/start.sh"] ``` I can follow this TIL: https://til.simonwillison.net/cloudrun/ship-dockerfile-to-cloud-run

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
Deploy a live instance of demos/apache-proxy 1058896236  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Queries took 21.335ms · About: github-to-sqlite