issue_comments

28 rows where issue = 608058890 sorted by updated_at descending

View and edit SQL

Suggested facets: reactions, created_at (date), updated_at (date)

user

author_association

issue

  • link_or_copy_directory() error - Invalid cross-device link · 28
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions issue
635386935 https://github.com/simonw/datasette/issues/744#issuecomment-635386935 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNTM4NjkzNQ== aborruso 30607 2020-05-28T14:32:53Z 2020-05-28T14:32:53Z NONE

Wow, I'm in some way very proud!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
635384739 https://github.com/simonw/datasette/issues/744#issuecomment-635384739 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNTM4NDczOQ== simonw 9599 2020-05-28T14:28:58Z 2020-05-28T14:28:58Z OWNER

This is now released. pip install datasette==0.43 should get the fix.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
634850676 https://github.com/simonw/datasette/issues/744#issuecomment-634850676 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNDg1MDY3Ng== simonw 9599 2020-05-27T18:18:01Z 2020-05-27T18:18:01Z OWNER

Thanks for helping test this @aborruso!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
634446887 https://github.com/simonw/datasette/issues/744#issuecomment-634446887 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNDQ0Njg4Nw== aborruso 30607 2020-05-27T06:01:28Z 2020-05-27T06:01:28Z NONE

Dear @simonw thank you for your time, now IT WORKS!!!

I hope that this edit to datasette code is not for an exceptional case (my PC configuration) and that it will be useful to other users.

Thank you again!!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
634395343 https://github.com/simonw/datasette/issues/744#issuecomment-634395343 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNDM5NTM0Mw== simonw 9599 2020-05-27T02:49:26Z 2020-05-27T02:49:26Z OWNER

OK, here's a new branch you can try. Install it like this:

pip install https://github.com/simonw/datasette/archive/shutil-backport.zip

If it works for you I'll merge that branch into master.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
634283355 https://github.com/simonw/datasette/issues/744#issuecomment-634283355 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNDI4MzM1NQ== aborruso 30607 2020-05-26T21:15:34Z 2020-05-26T21:15:34Z NONE

Oh no! It looks like dirs_exist_ok is Python 3.8 only. This is a bad fix, it needs to work on older Python's too. Re-opening.

Thank you very much

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
634254943 https://github.com/simonw/datasette/issues/744#issuecomment-634254943 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNDI1NDk0Mw== simonw 9599 2020-05-26T20:15:11Z 2020-05-26T20:15:11Z OWNER

Oh no! It looks like dirs_exist_ok is Python 3.8 only. This is a bad fix, it needs to work on older Python's too. Re-opening.

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 1,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
634254700 https://github.com/simonw/datasette/issues/744#issuecomment-634254700 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzNDI1NDcwMA== simonw 9599 2020-05-26T20:14:35Z 2020-05-26T20:14:35Z OWNER

What version of Python are you running?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
632305868 https://github.com/simonw/datasette/issues/744#issuecomment-632305868 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzMjMwNTg2OA== aborruso 30607 2020-05-21T19:43:23Z 2020-05-21T19:43:23Z NONE

@simonw now I have

Traceback (most recent call last):
  File "/home/aborruso/.local/bin/datasette", line 8, in <module>
    sys.exit(cli())
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 103, in heroku
    extra_metadata,
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 191, in temporary_heroku_directory
    os.path.join(tmp.name, "templates"),
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 605, in link_or_copy_directory
    shutil.copytree(src, dst, copy_function=os.link, dirs_exist_ok=True)
TypeError: copytree() got an unexpected keyword argument 'dirs_exist_ok'

Do I must open a new issue?

Thank you

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
632255088 https://github.com/simonw/datasette/issues/744#issuecomment-632255088 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzMjI1NTA4OA== aborruso 30607 2020-05-21T17:58:51Z 2020-05-21T17:58:51Z NONE

Thank you very much!!

I will try and I write you here

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
632253073 https://github.com/simonw/datasette/issues/744#issuecomment-632253073 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzMjI1MzA3Mw== simonw 9599 2020-05-21T17:54:55Z 2020-05-21T17:54:55Z OWNER

I think this is the fix! I just landed it in master.

@aborruso you can install Datasette master to test it out like this:

pip install https://github.com/simonw/datasette/archive/cee671a58f417f827d1735b1abaa40716534ea67.zip
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
632249565 https://github.com/simonw/datasette/issues/744#issuecomment-632249565 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzMjI0OTU2NQ== aborruso 30607 2020-05-21T17:47:40Z 2020-05-21T17:47:40Z NONE

@simonw can I test it know? What I must do to update it?

Thank you

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
632237195 https://github.com/simonw/datasette/issues/744#issuecomment-632237195 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzMjIzNzE5NQ== simonw 9599 2020-05-21T17:23:39Z 2020-05-21T17:23:48Z OWNER

Actually maybe the answer here is to use dirs_exist_ok=True when calling shutil.copytree.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
632233393 https://github.com/simonw/datasette/issues/744#issuecomment-632233393 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYzMjIzMzM5Mw== simonw 9599 2020-05-21T17:16:15Z 2020-05-21T17:17:43Z OWNER

I just hit this bug myself, or at least a variant of it!

https://github.com/simonw/museums/runs/697063068?check_suite_focus=true

2020-05-21T17:14:02.2596471Z Traceback (most recent call last):
2020-05-21T17:14:02.2599146Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/utils/__init__.py", line 605, in link_or_copy_directory
2020-05-21T17:14:02.2599861Z     shutil.copytree(src, dst, copy_function=os.link)
2020-05-21T17:14:02.2600377Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 554, in copytree
2020-05-21T17:14:02.2600786Z     return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
2020-05-21T17:14:02.2601196Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 510, in _copytree
2020-05-21T17:14:02.2601580Z     raise Error(errors)
2020-05-21T17:14:02.2604565Z shutil.Error: [('/home/runner/work/museums/museums/templates/row-browse-museums.html', '/tmp/tmpq47xi96y/datasette/templates/row-browse-museums.html',
"[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/row-browse-museums.html' -> '/tmp/tmpq47xi96y/datasette/templates/row-browse-museums.html'"), ('/home/runner/work/museums/museums/templates/index.html', '/tmp/tmpq47xi96y/datasette/templates/index.html',
"[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/index.html' -> '/tmp/tmpq47xi96y/datasette/templates/index.html'"), ('/home/runner/work/museums/museums/templates/query-browse-search.html', '/tmp/tmpq47xi96y/datasette/templates/query-browse-search.html',
"[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/query-browse-search.html' -> '/tmp/tmpq47xi96y/datasette/templates/query-browse-search.html'"), ('/home/runner/work/museums/museums/templates/_analytics.html', '/tmp/tmpq47xi96y/datasette/templates/_analytics.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/_analytics.html' -> '/tmp/tmpq47xi96y/datasette/templates/_analytics.html'"), ('/home/runner/work/museums/museums/templates/_museum_card.html', '/tmp/tmpq47xi96y/datasette/templates/_museum_card.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/_museum_card.html' -> '/tmp/tmpq47xi96y/datasette/templates/_museum_card.html'"), ('/home/runner/work/museums/museums/templates/pages/about.html', '/tmp/tmpq47xi96y/datasette/templates/pages/about.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/pages/about.html' -> '/tmp/tmpq47xi96y/datasette/templates/pages/about.html'"), ('/home/runner/work/museums/museums/templates/pages/map.html', '/tmp/tmpq47xi96y/datasette/templates/pages/map.html', "[Errno 18] Invalid cross-device link: '/home/runner/work/museums/museums/templates/pages/map.html' -> '/tmp/tmpq47xi96y/datasette/templates/pages/map.html'")]
2020-05-21T17:14:02.2605437Z 
2020-05-21T17:14:02.2605797Z During handling of the above exception, another exception occurred:
2020-05-21T17:14:02.2606102Z 
2020-05-21T17:14:02.2606423Z Traceback (most recent call last):
2020-05-21T17:14:02.2606817Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/bin/datasette", line 8, in <module>
2020-05-21T17:14:02.2607189Z     sys.exit(cli())
2020-05-21T17:14:02.2607907Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 829, in __call__
2020-05-21T17:14:02.2608347Z     return self.main(*args, **kwargs)
2020-05-21T17:14:02.2609024Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 782, in main
2020-05-21T17:14:02.2609451Z     rv = self.invoke(ctx)
2020-05-21T17:14:02.2610116Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
2020-05-21T17:14:02.2610550Z     return _process_result(sub_ctx.command.invoke(sub_ctx))
2020-05-21T17:14:02.2611451Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
2020-05-21T17:14:02.2611989Z     return _process_result(sub_ctx.command.invoke(sub_ctx))
2020-05-21T17:14:02.2612682Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
2020-05-21T17:14:02.2613117Z     return ctx.invoke(self.callback, **ctx.params)
2020-05-21T17:14:02.2613801Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/click/core.py", line 610, in invoke
2020-05-21T17:14:02.2614235Z     return callback(*args, **kwargs)
2020-05-21T17:14:02.2614946Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/publish/cloudrun.py", line 111, in cloudrun
2020-05-21T17:14:02.2615371Z     with temporary_docker_directory(
2020-05-21T17:14:02.2615769Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/contextlib.py", line 113, in __enter__
2020-05-21T17:14:02.2616135Z     return next(self.gen)
2020-05-21T17:14:02.2616957Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/utils/__init__.py", line 392, in temporary_docker_directory
2020-05-21T17:14:02.2617403Z     link_or_copy_directory(
2020-05-21T17:14:02.2618298Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/site-packages/datasette/utils/__init__.py", line 607, in link_or_copy_directory
2020-05-21T17:14:02.2619270Z     shutil.copytree(src, dst)
2020-05-21T17:14:02.2619872Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 554, in copytree
2020-05-21T17:14:02.2622179Z     return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
2020-05-21T17:14:02.2622609Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/shutil.py", line 455, in _copytree
2020-05-21T17:14:02.2622878Z     os.makedirs(dst, exist_ok=dirs_exist_ok)
2020-05-21T17:14:02.2623157Z   File "/opt/hostedtoolcache/Python/3.8.3/x64/lib/python3.8/os.py", line 223, in makedirs
2020-05-21T17:14:02.2623405Z     mkdir(name, mode)
2020-05-21T17:14:02.2623978Z FileExistsError: [Errno 17] File exists: '/tmp/tmpq47xi96y/datasette/templates'
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
625091976 https://github.com/simonw/datasette/issues/744#issuecomment-625091976 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyNTA5MTk3Ng== aborruso 30607 2020-05-07T07:51:25Z 2020-05-07T07:51:25Z NONE

I have installed heroku plugins:install heroku-builds, but I have the same error.

Then I have removed from datasette\publish\heroku.py

        # Check for heroku-builds plugin
        plugins = [
            line.split()[0] for line in check_output(["heroku", "plugins"]).splitlines()
        ]
        if b"heroku-builds" not in plugins:
            click.echo(
                "Publishing to Heroku requires the heroku-builds plugin to be installed."
            )
            click.confirm(
                "Install it? (this will run `heroku plugins:install heroku-builds`)",
                abort=True,
            )
            call(["heroku", "plugins:install", "heroku-builds"])

And now I have

Traceback (most recent call last):
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 210, in temporary_heroku_directory
    yield
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 96, in heroku
    list_output = check_output(["heroku", "apps:list", "--json"]).decode(
  File "c:\python37\lib\subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "c:\python37\lib\subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "c:\python37\lib\subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "c:\python37\lib\subprocess.py", line 1178, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The specified file could not be found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\Scripts\datasette.exe\__main__.py", line 9, in <module>
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 120, in heroku
    call(["heroku", "builds:create", "-a", app_name, "--include-vcs-ignore"])
  File "c:\python37\lib\contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 213, in temporary_heroku_directory
    tmp.cleanup()
  File "c:\python37\lib\tempfile.py", line 809, in cleanup
    _shutil.rmtree(self.name)
  File "c:\python37\lib\shutil.py", line 513, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\python37\lib\shutil.py", line 401, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "c:\python37\lib\shutil.py", line 399, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] Unable to access file. The file is being used by another process: 'C:\\Users\\aborr\\AppData\\Local\\Temp\\tmpkcxy8i_q'
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
625083715 https://github.com/simonw/datasette/issues/744#issuecomment-625083715 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyNTA4MzcxNQ== aborruso 30607 2020-05-07T07:34:18Z 2020-05-07T07:34:18Z NONE

In Windows I'm not very strong. I use debian (inside WSL).

However these are the possible steps:

  • I have installed Python 3 for win (I have 3.7.3);
  • I have installed heroku cli for win64 and logged in;
  • I have installed datasette running python -m pip install --upgrade --user datasette.

It's a very basic Python env that I do not use. This time only to reach my goal: try to publish using custom template

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
625066876 https://github.com/simonw/datasette/issues/744#issuecomment-625066876 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyNTA2Njg3Ng== simonw 9599 2020-05-07T06:55:04Z 2020-05-07T06:55:04Z OWNER

I have a Windows 10 gaming PC in the house: what steps can I take to get an environment in there that matches yours? I've not installed Python on Windows before.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
625066073 https://github.com/simonw/datasette/issues/744#issuecomment-625066073 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyNTA2NjA3Mw== aborruso 30607 2020-05-07T06:53:09Z 2020-05-07T06:53:09Z NONE

@simonw another error starting from Windows.

I run

datasette  publish heroku -n comunepa --template-dir template commissioniComunePalermo.db

And I have

Traceback (most recent call last):
  File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\Scripts\datasette.exe\__main__.py", line 9, in <module>
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\aborr\AppData\Roaming\Python\Python37\site-packages\datasette\publish\heroku.py", line 53, in heroku
    line.split()[0] for line in check_output(["heroku", "plugins"]).splitlines()
  File "c:\python37\lib\subprocess.py", line 395, in check_output
    **kwargs).stdout
  File "c:\python37\lib\subprocess.py", line 472, in run
    with Popen(*popenargs, **kwargs) as process:
  File "c:\python37\lib\subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "c:\python37\lib\subprocess.py", line 1178, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The specified file could not be found

files.zip

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
625061250 https://github.com/simonw/datasette/issues/744#issuecomment-625061250 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyNTA2MTI1MA== simonw 9599 2020-05-07T06:40:09Z 2020-05-07T06:40:09Z OWNER

Have a try and let me know what happens! I'd like this stuff to just work on Windows but I need to figure out how to get an environment working.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
625060561 https://github.com/simonw/datasette/issues/744#issuecomment-625060561 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyNTA2MDU2MQ== aborruso 30607 2020-05-07T06:38:24Z 2020-05-07T06:38:24Z NONE

Hi @simonw probably I could try to do it in Python for windows. I do not like to do these things in win enviroment.

Because probably WSL Linux env (in which I do a lot of great things) is not an environment that will be tested for datasette.

In win I shouldn't have any problems. Am I right?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
621030783 https://github.com/simonw/datasette/issues/744#issuecomment-621030783 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMTAzMDc4Mw== aborruso 30607 2020-04-29T07:16:27Z 2020-04-29T07:16:27Z NONE

Hi @simonw it's debian as Windows Subsystem for Linux

PRETTY_NAME="Pengwin"
NAME="Pengwin"
VERSION_ID="10"
VERSION="10 (buster)"
ID=debian
ID_LIKE=debian
HOME_URL="https://github.com/whitewaterfoundry/Pengwin"
SUPPORT_URL="https://github.com/whitewaterfoundry/Pengwin"
BUG_REPORT_URL="https://github.com/whitewaterfoundry/Pengwin"
VERSION_CODENAME=buster
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
621027697 https://github.com/simonw/datasette/issues/744#issuecomment-621027697 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMTAyNzY5Nw== simonw 9599 2020-04-29T07:08:09Z 2020-04-29T07:08:09Z OWNER

What operating system are you using? I need to figure out a way to replicate this bug.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
621011554 https://github.com/simonw/datasette/issues/744#issuecomment-621011554 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMTAxMTU1NA== aborruso 30607 2020-04-29T06:17:26Z 2020-04-29T06:17:26Z NONE

A stupid note: I have no tmpcqv_1i5d folder in in /tmp.

It seems to me that it does not create any /tmp/tmpcqv_1i5d/templates folder (or other name folder, inside /tmp)

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
621008152 https://github.com/simonw/datasette/issues/744#issuecomment-621008152 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMTAwODE1Mg== aborruso 30607 2020-04-29T06:05:02Z 2020-04-29T06:05:02Z NONE

Hi @simonw , I have installed it and I have the below errors.

Is it possible that your /tmp directory is on a different volume from the template folder? That could cause a problem with the symlinks.

No, /tmp folder is in the same volume.

Thank you

Traceback (most recent call last):
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 607, in link_or_copy_directory
    shutil.copytree(src, dst, copy_function=os.link)
  File "/usr/lib/python3.7/shutil.py", line 365, in copytree
    raise Error(errors)
shutil.Error: [('/var/youtubeComunePalermo/processing/./template/base.html', '/tmp/tmpcqv_1i5d/templates/base.html', "[Errno 18] Invalid cross-device link: '/var/youtubeComunePalermo/processing/./template/base.html' -> '/tmp/tmpcqv_1i5d/templates/base.html'"), ('/var/youtubeComunePalermo/processing/./template/index.html', '/tmp/tmpcqv_1i5d/templates/index.html', "[Errno 18] Invalid cross-device link: '/var/youtubeComunePalermo/processing/./template/index.html' -> '/tmp/tmpcqv_1i5d/templates/index.html'")]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aborruso/.local/bin/datasette", line 8, in <module>
    sys.exit(cli())
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/aborruso/.local/lib/python3.7/site-packages/click/core.py", line 610, in invoke    return callback(*args, **kwargs)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 103, in heroku
    extra_metadata,
  File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
    return next(self.gen)
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/publish/heroku.py", line 191, in temporary_heroku_directory
    os.path.join(tmp.name, "templates"),
  File "/home/aborruso/.local/lib/python3.7/site-packages/datasette/utils/__init__.py", line 609, in link_or_copy_directory
    shutil.copytree(src, dst)
  File "/usr/lib/python3.7/shutil.py", line 321, in copytree
    os.makedirs(dst)
  File "/usr/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/tmp/tmpcqv_1i5d/templates'
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
620971526 https://github.com/simonw/datasette/issues/744#issuecomment-620971526 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMDk3MTUyNg== simonw 9599 2020-04-29T03:32:14Z 2020-04-29T03:32:14Z OWNER

@aborruso I think I have a branch with a fix - could you try it out?

Install the new branch like this:

pip install https://github.com/simonw/datasette/archive/issue-744.zip

Then try running datasette publish heroku again.

If this fixes your bug I'll merge that pull request!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
620970475 https://github.com/simonw/datasette/issues/744#issuecomment-620970475 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMDk3MDQ3NQ== simonw 9599 2020-04-29T03:27:20Z 2020-04-29T03:27:20Z OWNER

This exception handling was introduced in #141

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
620970159 https://github.com/simonw/datasette/issues/744#issuecomment-620970159 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMDk3MDE1OQ== simonw 9599 2020-04-29T03:25:45Z 2020-04-29T03:25:45Z OWNER

https://docs.python.org/3/library/shutil.html#shutil.copytree says that shutil.Error should be raised here, so I think this is a bug in the link_or_copy_directory function.

I don't have an environment that can replicate this bug. @aborruso I'm going to ship a fix in a branch that you can test against.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890
620969639 https://github.com/simonw/datasette/issues/744#issuecomment-620969639 https://api.github.com/repos/simonw/datasette/issues/744 MDEyOklzc3VlQ29tbWVudDYyMDk2OTYzOQ== simonw 9599 2020-04-29T03:23:31Z 2020-04-29T03:23:31Z OWNER

Is it possible that your /tmp directory is on a different volume from the template folder? That could cause a problem with the symlinks.

Here's the code in question:

https://github.com/simonw/datasette/blob/89c4ddd4828623888e91a1d2cb396cba12d4e7b4/datasette/utils/init.py#L595-L609

It looks to me like we were expecting an OSError but in your environment you got a shutil.Error instead. So a fix could be to capture that error as well.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
link_or_copy_directory() error - Invalid cross-device link 608058890

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
                ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
                ON [issue_comments] ([user]);
Powered by Datasette · Query took 30.016ms · About: github-to-sqlite