issue_comments

Uvicorn have a release out now that would have fixed this issue if I hadn't shipped my own fix: https://github.com/encode/uvicorn/releases/tag/0.17.0.post1

Just shipped 0.60.1 with the fix - and tested that pip install datasette does indeed work correctly on Python 3.6.

So I don't need to release 0.60.1 AND 0.60.2 after all - I can just release 0.60.1 with a bug fix that it no longer breaks installation for Python 3.6.

Out of curiosity, I installed this latest setup.py file using both Python 3.6 and Python 3.10, ran pip freeze on both of them and created a Gist to compare the difference. The result is here: https://gist.github.com/simonw/2e7d5b1beba675ef9a5bcd310cadc372/revisions

From that, it looks like the Python packages in my dependencies which have released new versions that don't work with Python 3.6 are:

• https://pypi.org/project/janus/#history
• https://pypi.org/project/Pint/#history
• https://pypi.org/project/platformdirs/#history
• https://pypi.org/project/uvicorn/#history (already discussed)

Sure enough, for the first three of those browsing through their recent versions on PyPI confirms that they switched from e.g. "Requires: Python >=3.6" on https://pypi.org/project/janus/0.7.0/ to "Requires: Python >=3.7" on https://pypi.org/project/janus/1.0.0/

OK, the environment markers approach seems to work!

https://github.com/simonw/datasette/runs/4890775227?check_suite_focus=true - the tests passed on Python 3.6 for this commit with the pinned dependencies: https://github.com/simonw/datasette/commit/41060e7e7cb838328c879de6a98ae794dc1886d0

This whole thing reminds me of my ongoing internal debate about version pinning: should the Datasette package released to PyPI pin to the exact versions of the dependencies that are known to work, or should it allow a range of dependencies so users can pick other versions of the dependencies to use in their environment?

As I understand it, the general rule is to use exact pinning for applications but use ranges for libraries.

Datasette is almost entirely an application... but it can also be used as a library - and in fact I'm hoping to encourage that usage more in the future, see: - #1398

I'd also like to release a packaged version of Datasette that doesn't require Uvicorn, for running on AWS Lambda and other function-as-a-service platforms. Those platforms have their own HTTP layer and hence don't need the Uvicorn dependency.

Maybe the answer is to have a datasette-core package which provides the core of Datasette with unpinned dependencies and no Uvicorn, and then have the existing datasette package provide the Datasette CLI tool with Uvicorn and pinned dependencies?

Another option from https://twitter.com/samuel_hames/status/1484327636860293121 - environment markers, described in https://www.python.org/dev/peps/pep-0508/#environment-markers

Found some examples of those in use using GitHub code search: https://cs.github.com/?scopeName=All+repos&scope=&q=%22%3Bpython_version%22+path%3Asetup.py - in particular https://github.com/xmendez/wfuzz/blob/1b695ee9a87d66a7d7bf6cae70d60a33fae51541/setup.py#L31-L38

python install_requires = [ 'pycurl', 'pyparsing<2.4.2;python_version<="3.4"', 'pyparsing>=2.4*;python_version>="3.5"', 'six', 'configparser;python_version<"3.5"', 'chardet', ] So maybe I can ship 0.60.1 with loose dependencies except for the uvicorn one on Python 3.6, using an environment marker.

Here's my setup.py at the moment: https://github.com/simonw/datasette/blob/ffca55dfd7cc9b53522c2e5a2fa1ff67c9beadf2/setup.py#L44-L61

One other problem: there might be packages in that list right now which don't specify their 3.6 Python version but which will, at some point in the future, release a new version that doesn't work with 3.6 (like Uvicorn did) - in which case Python 3.6 installs would break in the future.

I think what I'll do then is ship the 0.60.1 Python 3.6 version with strict upper limits on each version which are the current, tested-with-Datasette-on-Python3.6 versions.

Problem: if I ship this, it will be the most recent release of Datasette - but unlike other previous releases it has exactly pinned versions of all of the dependencies. Which is bad for people who run pip install datasette but want to not be stuck to those exact library versions.

So maybe I ship this as 0.60.1, then ship a 0.60.2 release directly afterwards which unpins the dependencies again and requires Python 3.7?

I used the combo of pyenv and pipenv to run tests and figure out what the most recent versions of each dependency were that worked on Python 3.6. I also clicked around in the latest releases on pages such as https://pypi.org/project/aiofiles

cd /tmp git clone git@github.com:simonw/datasette cd /tmp/datasette pipenv shell --python 3.6.10 pip install -e '.[test]' pytest I also used pip freeze | grep black to see which version was installed, since packages with python_requires= in them would automatically install the highest compatible version.

I think there are two possible solutions then:

1. Convince Uvicorn to publish one last 0.16.1 version which includes that python_requires= line, such that there's a version of Uvicorn on PyPI that Python 3.6 can still install.
2. Release a 0.60.1 version of Datasette which pins that Uvicorn version, and hence can be installed.

I've made the request for 1) in Uvicorn Gitter here: https://gitter.im/encode/community?at=61ea044a6d9ba23328d0fa28

I'm going to investigate option 2) myself now.

On Twitter: https://twitter.com/simonw/status/1484317711672877065

Here's the problem: Uvicorn only added python_requires to their setup.py a few days ago, which means the releases they have out on PyPI at the moment don't specify the Python version they need, which is why this mechanism doesn't work as expected: - https://github.com/encode/uvicorn/pull/1328