22 rows where issue = 742011049 sorted by updated_at descending

View and edit SQL

Suggested facets: reactions, created_at (date), updated_at (date)

user

author_association

issue

  • .json and .csv exports fail to apply base_url · 22
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue performed_via_github_app
758668359 https://github.com/simonw/datasette/issues/1091#issuecomment-758668359 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc1ODY2ODM1OQ== tballison 6739646 2021-01-12T13:52:29Z 2021-01-12T13:52:29Z NONE

Y, thank you to both of you!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
758448525 https://github.com/simonw/datasette/issues/1091#issuecomment-758448525 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc1ODQ0ODUyNQ== henry501 19328961 2021-01-12T06:55:08Z 2021-01-12T06:55:08Z NONE

Great, really happy I could help! Reverse proxies get tricky.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
758283074 https://github.com/simonw/datasette/issues/1091#issuecomment-758283074 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc1ODI4MzA3NA== simonw 9599 2021-01-11T23:12:46Z 2021-01-11T23:12:46Z OWNER

Fantastic!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
758280611 https://github.com/simonw/datasette/issues/1091#issuecomment-758280611 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc1ODI4MDYxMQ== tballison 6739646 2021-01-11T23:06:10Z 2021-01-11T23:06:10Z NONE

+1

Yep! Fixes it. If I navigate to https://corpora.tika.apache.org/datasette, I get a 404 (database not found: datasette), but if I navigate to https://corpora.tika.apache.org/datasette/file_profiles/, everything WORKS!

Thank you!

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 1,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
756453945 https://github.com/simonw/datasette/issues/1091#issuecomment-756453945 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc1NjQ1Mzk0NQ== simonw 9599 2021-01-07T23:42:50Z 2021-01-07T23:42:50Z OWNER

@henry501 it looks like you spotted a bug in the documentation - I just addressed that, the fix is now live here: https://docs.datasette.io/en/latest/deploying.html#running-datasette-behind-a-proxy

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
756453010 https://github.com/simonw/datasette/issues/1091#issuecomment-756453010 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc1NjQ1MzAxMA== simonw 9599 2021-01-07T23:39:58Z 2021-01-07T23:40:25Z OWNER

@tballison I think that's the solution! It looks like you need to use this in your config:

ProxyPass /datasette http://127.0.0.1:8001/datasette

Instead of this:

ProxyPass /datasette http://127.0.0.1:8001/

Give that a go and let me know if it fixes it.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
756425587 https://github.com/simonw/datasette/issues/1091#issuecomment-756425587 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc1NjQyNTU4Nw== henry501 19328961 2021-01-07T22:27:19Z 2021-01-07T22:27:19Z NONE

I found this issue while troubleshooting the same behavior with an nginx reverse proxy. The solution was to make sure I set:

proxy_pass http://server:8001/baseurl/
instead of just:

proxy_pass http://server:8001
The custom SQL query and header links are now correct.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
741992106 https://github.com/simonw/datasette/issues/1091#issuecomment-741992106 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc0MTk5MjEwNg== simonw 9599 2020-12-09T19:19:54Z 2020-12-09T20:27:45Z OWNER

Could you try removing the ProxyPassReverse /datasette http://0.0.0.0:8001 line?

My hunch is that ProxyPassReverse is rewriting some of the links in the HTML (or maybe in the HTTP headers) in a way that breaks things.

Normally you would need ProxyPassReverse to compensate for the underlying application being unable to rewrite its links - but Datasette's base_url setting causes Datasette to rewrite all of the links for you, so ProxyPassReverse should be unneccessary.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
742023541 https://github.com/simonw/datasette/issues/1091#issuecomment-742023541 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc0MjAyMzU0MQ== simonw 9599 2020-12-09T20:17:54Z 2020-12-09T20:17:54Z OWNER

OK that is really weird. I'll have another go at replicating this locally.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
742010306 https://github.com/simonw/datasette/issues/1091#issuecomment-742010306 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc0MjAxMDMwNg== tballison 6739646 2020-12-09T19:53:18Z 2020-12-09T19:59:52Z NONE

I can't imagine this helps (esp. given your point about potential rewrites), but you can see that /datasette/ was correctly added to the sql form, but not to the "export-links"

https://user-images.githubusercontent.com/6739646/101680055-234baa00-3a2e-11eb-8650-2b369bc6f031.png">

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
742001510 https://github.com/simonw/datasette/issues/1091#issuecomment-742001510 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc0MjAwMTUxMA== tballison 6739646 2020-12-09T19:36:42Z 2020-12-09T19:38:04Z NONE

I don't think this fixes it:

grep -R datasette .
./sites-available/000-default.conf:        ProxyPass /datasette http://127.0.0.1:8001/
./sites-available/000-default.conf:        #ProxyPassReverse /datasette http://127.0.0.1:8001/
./sites-available/corpora-le-ssl.conf:  ProxyPass /datasette http://0.0.0.0:8001
./sites-available/corpora-le-ssl.conf:  #ProxyPassReverse /datasette http://0.0.0.0:8001
./sites-enabled/corpora-le-ssl.conf:  ProxyPass /datasette http://0.0.0.0:8001
./sites-enabled/corpora-le-ssl.conf:  #ProxyPassReverse /datasette http://0.0.0.0:8001

And I confirmed that I actually restarted the server. :rofl:

https://corpora.tika.apache.org/datasette/file_profiles

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
741804334 https://github.com/simonw/datasette/issues/1091#issuecomment-741804334 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDc0MTgwNDMzNA== tballison 6739646 2020-12-09T14:26:05Z 2020-12-09T14:26:05Z NONE

Anything we can do to help debug this? Thank you, again!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
729045320 https://github.com/simonw/datasette/issues/1091#issuecomment-729045320 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyOTA0NTMyMA== tballison 6739646 2020-11-17T16:31:00Z 2020-11-17T16:31:00Z NONE

We're using mod_proxy.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
729018386 https://github.com/simonw/datasette/issues/1091#issuecomment-729018386 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyOTAxODM4Ng== tballison 6739646 2020-11-17T15:48:58Z 2020-11-17T15:48:58Z NONE

I don't think we are, but I'll check with Maruan.

I think this is the relevant part of our config?

  Alias "/base/" "/usr/share/corpora/"
  <Directory "/usr/share/corpora/">
    Options +Indexes -Multiviews
    AllowOverride None
  </Directory>

  ProxyPreserveHost On

  ProxyPass /datasette http://0.0.0.0:8001
  ProxyPassReverse /datasette http://0.0.0.0:8001

</VirtualHost>
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
728262974 https://github.com/simonw/datasette/issues/1091#issuecomment-728262974 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyODI2Mjk3NA== simonw 9599 2020-11-16T19:05:08Z 2020-11-16T19:05:08Z OWNER

I have a hunch that there may be some extra configuration in play here - could Apache itself be rewriting some of the links using mod_proxy_html?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
727233553 https://github.com/simonw/datasette/issues/1091#issuecomment-727233553 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyNzIzMzU1Mw== simonw 9599 2020-11-14T16:46:52Z 2020-11-14T16:46:52Z OWNER

@tballison could I see the section of your Apache config that configures the proxying to /datasette/?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
726801731 https://github.com/simonw/datasette/issues/1091#issuecomment-726801731 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyNjgwMTczMQ== tballison 6739646 2020-11-13T14:40:56Z 2020-11-13T14:40:56Z NONE

My headers aren't clickable/sortable with custom sql, but I think that's by design.

In the default view, https://corpora.tika.apache.org/datasette/file_profiles/file_profiles, ah, y, now I see that the headers should be sortable, but you're right the base_url is not applied.

base_url works with "View and Edit SQL" and with "(advanced)"

As you point out, does not work with the export csv, json, other or with the "Next page" navigational button at the bottom.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
726798745 https://github.com/simonw/datasette/issues/1091#issuecomment-726798745 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyNjc5ODc0NQ== tballison 6739646 2020-11-13T14:35:22Z 2020-11-13T14:35:22Z NONE

I'm starting this with docker like so:

docker run --name datasette -d -p 8001:8001 -vpwd:/mnt datasetteproject/datasette datasette -p 8001 -h 0.0.0.0 /mnt/file_profiles.db --config sql_time_limit_ms:120000 --config max_returned_rows:100000 --config base_url:/datasette/ --config cache_size_kb:50000

I'm not doing any templating or anything else custom.

Apropos of nothing, I swapped out a simpler db, so this query should now work:

https://corpora.tika.apache.org/datasette/file_profiles?sql=select%0D%0A++*%0D%0Afrom%0D%0A++file_profiles+fp%0D%0Alimit%0D%0A++10

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
726416330 https://github.com/simonw/datasette/issues/1091#issuecomment-726416330 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyNjQxNjMzMA== simonw 9599 2020-11-13T00:00:43Z 2020-11-13T00:00:43Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
726415991 https://github.com/simonw/datasette/issues/1091#issuecomment-726415991 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyNjQxNTk5MQ== simonw 9599 2020-11-12T23:59:34Z 2020-11-12T23:59:34Z OWNER
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
726415019 https://github.com/simonw/datasette/issues/1091#issuecomment-726415019 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyNjQxNTAxOQ== simonw 9599 2020-11-12T23:56:23Z 2020-11-12T23:56:23Z OWNER

@tballison is there any chance you're running any custom templates in that installation? I'm really confused as to why I can't replicate the bug.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  
726413829 https://github.com/simonw/datasette/issues/1091#issuecomment-726413829 https://api.github.com/repos/simonw/datasette/issues/1091 MDEyOklzc3VlQ29tbWVudDcyNjQxMzgyOQ== simonw 9599 2020-11-12T23:52:50Z 2020-11-12T23:54:16Z OWNER

Hmm... it's not just the .csv and .json export links - it's the column headings (which can be clicked to change the sort order) as well. Here's an extract of the HTML from that page:

<p class="export-links">This data as 
  <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json">json</a>, 
  <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.csv?_size=max">CSV</a> (
  <a href="#export">advanced</a>)
</p>
<div class="table-wrapper">
  <table class="rows-and-columns">
    <thead>
      <tr>
        <th class="col-Link" scope="col" data-column="Link" data-column-type="" data-column-not-null="0" data-is-pk="0">
          Link
        </th>
        <th class="col-rowid" scope="col" data-column="rowid" data-column-type="integer" data-column-not-null="0" data-is-pk="1">
          <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES?_sort_desc=rowid" rel="nofollow">rowid&nbsp;▼</a>
        </th>
        <th class="col-PARSE_EXCEPTION_ID" scope="col" data-column="PARSE_EXCEPTION_ID" data-column-type="INTEGER" data-column-not-null="0" data-is-pk="0">
          <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES?_sort=PARSE_EXCEPTION_ID" rel="nofollow">PARSE_EXCEPTION_ID</a>
        </th>
        <th class="col-PARSE_EXCEPTION_DESCRIPTION" scope="col" data-column="PARSE_EXCEPTION_DESCRIPTION" data-column-type="VARCHAR(128)" data-column-not-null="0" data-is-pk="0">
          <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES?_sort=PARSE_EXCEPTION_DESCRIPTION" rel="nofollow">PARSE_EXCEPTION_DESCRIPTION</a>
        </th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td class="col-Link type-pk">
          <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/1">1</a>
        </td>
        <td class="col-rowid type-int">1</td>
        <td class="col-PARSE_EXCEPTION_ID type-int">0</td>
        <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">RUNTIME</td>
      </tr>
      <tr>
        <td class="col-Link type-pk">
          <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/2">2</a>
        </td>
        <td class="col-rowid type-int">2</td>
        <td class="col-PARSE_EXCEPTION_ID type-int">1</td>
        <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">ENCRYPTION</td>
      </tr>
      <tr>
        <td class="col-Link type-pk">
          <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/3">3</a>
        </td>
        <td class="col-rowid type-int">3</td>
        <td class="col-PARSE_EXCEPTION_ID type-int">2</td>
        <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">ACCESS_PERMISSION</td>
      </tr>
      <tr>
        <td class="col-Link type-pk">
          <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/4">4</a>
        </td>
        <td class="col-rowid type-int">4</td>
        <td class="col-PARSE_EXCEPTION_ID type-int">3</td>
        <td class="col-PARSE_EXCEPTION_DESCRIPTION type-str">UNSUPPORTED_VERSION</td>
      </tr>
    </tbody>
  </table>
</div>
<div id="export" class="advanced-export">
  <h3>Advanced export</h3>
  <p>JSON shape:
    <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json">default</a>,
    <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json?_shape=array">array</a>,
    <a href="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.json?_shape=array&amp;_nl=on">newline-delimited</a>
  </p>
  <form action="/corpora-metadata/REF_PARSE_EXCEPTION_TYPES.csv" method="get">
    <p>
      CSV options:            
      <label>
        <input type="checkbox" name="_dl"> download file
        </label>
        <input type="submit" value="Export CSV">
          <input type="hidden" name="_size" value="max">
          </p>
        </form>
      </div>

But here's something really weird - the links to the individual rows DO include the /datasette/ prefix:

<td class="col-Link type-pk">
    <a href="/datasette/corpora-metadata/REF_PARSE_EXCEPTION_TYPES/2">2</a>
</td>

The navigation bar on that page is correct too:

<p class="crumbs">
        <a href="/datasette/">home</a> /
        <a href="/datasette/corpora-metadata">corpora-metadata</a>
</p>

I've also been unable to replicate this in my own local environment, running datasette fixtures.db --config base_url:/datasette/.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
.json and .csv exports fail to apply base_url 742011049  

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
, [performed_via_github_app] TEXT);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);