python shutil.copy fails on fat file systems (ubuntu)

  • Last Update :
  • Techknowledgy :

If I know that the target is a file system where chmod fails, I simply delete the chmod method from the os package using del os.chmod, and this allows the copy to succeed.

>>>
import os
   >>>
   print hasattr(os, 'chmod')
True
   >>>
   foo = os.chmod >>>
   del os.chmod >>>
   print hasattr(os, 'chmod')
False

This now allows you to perform the copy without failing on the chmod. Then we re-enable it by assigning the attribute back.

>>> setattr(os, 'chmod', foo) >>>
   print hasattr(os, 'chmod')
True

Deleting os.chmod globally is not a good idea.

$ mkdir folder
$ touch folder / a
$ python2 .7 - c 'import shutil; shutil.copyfile("folder/a","folder/b")'
$ ls - rthla folder /
   total 0
drwxr - xr - x + Apr 17 12: 49. . /
   -rw - r--r--Apr 17 12: 49 a -
   rw - r--r--Apr 17 12: 50 b
drwxr - xr - x + Apr 17 12: 50 . /

As you can see in the python source code of shutil (/usr/lib/python2.7/shutil.py), there is no path consideration (relative/absolute) in the copy source code, the src variable is directly passed as an argument of copyfile.

def copy(src, dst):
   ""
"Copy data and mode bits ("
cp src dst ").

The destination may be a directory.

""
"
if os.path.isdir(dst):
   dst = os.path.join(dst, os.path.basename(src))
copyfile(src, dst)
copymode(src, dst)

Suggestion : 2

This issue tracker has been migrated to GitHub, and is currently read-only. For more information, see the GitHub FAQs in the Python's Developer Guide.

When calling `shutil.copy('file.txt', 'not_here/')`,
   if directory `not_here/`
does not exist,
the raised error is:

   IsADirectoryError: [Errno 21] Is a directory: 'not_here/'

If the intent of the user was to copy a file in a directory
but the user forgot to create the destination directory,
this can be very misleading,
as the error tends to indicate that the directory exists.

It happened to me and I was thinking
   "yes it's a directory, then what?
that 's exactly what I want,copy to this directory!"
when the problem was that I forgot to create the destination directory.

I would suggest to
catch the `IsADirectoryError` in shutil.copyfile()(at `open(dst, 'wb')`)
and raise instead an error saying something like
   "destination is a directory AND this directory does not exists".
Agreed this is confusing. Is this a Linux specific error? Trying this on Mac gives me a different error code and exception.

# Mac

$ ./python.exe
Python 3.8.0a0 (heads/master:cd449806fa, Nov 12 2018, 09:51:24)
[Clang 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> shutil.copy('/tmp/a.py', 'Lib12/')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
      File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/shutil.py", line 385, in copy
      copyfile(src, dst, follow_symlinks=follow_symlinks)
      File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/shutil.py", line 240, in copyfile
      with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
      FileNotFoundError: [Errno 2] No such file or directory: 'Lib12/'

      # Ubuntu

      ./python
      Python 3.8.0a0 (heads/master:dce345c51a, Nov 12 2018, 13:01:05)
      [GCC 5.4.0 20160609] on linux
      Type "help", "copyright", "credits" or "license" for more information.
      >>> import shutil
      >>> shutil.copy('/tmp/a.py', 'Lib12/')
      Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
            File "/home/karthi/cpython/Lib/shutil.py", line 386, in copy
            copyfile(src, dst, follow_symlinks=follow_symlinks)
            File "/home/karthi/cpython/Lib/shutil.py", line 241, in copyfile
            with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
            IsADirectoryError: [Errno 21] Is a directory: 'Lib12/'
You have the issue with the built - in 'open'
function.
This error is specific to the C-API and not to Python,

here is an example.

#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>

int main(int argc, char **argv, char **environ) {
    int fd;
    fd = open("/tmp/toto/", O_CREAT);
    printf("file descriptor: %d\n", fd);
    printf(strerror(errno));
    close(fd);
    return 0;
}



./a.out
file descriptor: -1
Is a directory
@vstinner & @serhiy

What do you think about this issue ?

   For the same POSIX syscall(open) we get 2 different values
for
the error, 2 on macOS and EISDIR with Linux.

Is a bug in Python or with the compliance of the operating system and the POSIX norm ?

   Thank you
Using `cp`
on Debian Buster I 'm having a better error message:

$ touch foo
$ cp foo bar /
   cp: failed to access 'bar/': Not a directory

From copy.c(from Debian coreutils):

   /* Improve quality of diagnostic when a nonexistent dst_name
      ends in a slash and open fails with errno == EISDIR.  */
   if (dest_desc < 0 && dest_errno == EISDIR &&
      * dst_name && dst_name[strlen(dst_name) - 1] == '/')
      dest_errno = ENOTDIR;

Suggestion : 3

All functions in this module raise OSError (or subclasses thereof) in the case of invalid or inaccessible file names and paths, or other arguments that have the correct type, but are not accepted by the operating system.,A set object indicating which functions in the os module permit specifying their path parameter as an open file descriptor on the local platform. Different platforms provide different features, and the underlying functionality Python uses to accept open file descriptors as path arguments is not available on all platforms Python supports.,Rename the file or directory src to dst. If dst exists, the operation will fail with an OSError subclass in a number of cases:,All functions accepting path or file names accept both bytes and string objects, and result in an object of the same type, if a path or file name is returned.

for fd in range(fd_low, fd_high):
   try:
   os.close(fd)
except OSError:
   pass
if os.access("myfile", os.R_OK):
   with open("myfile") as fp:
   return fp.read()
return "some default data"
try:
fp = open("myfile")
except PermissionError:
   return "some default data"
else:
   with fp:
   return fp.read()
with os.scandir(path) as it:
   for entry in it:
   if not entry.name.startswith('.') and entry.is_file():
   print(entry.name)
>>>
import os
   >>>
   statinfo = os.stat('somefile.txt') >>>
   statinfo
os.stat_result(st_mode = 33188, st_ino = 7876932, st_dev = 234881026,
      st_nlink = 1, st_uid = 501, st_gid = 501, st_size = 264, st_atime = 1297230295,
      st_mtime = 1297230027, st_ctime = 1297230027) >>>
   statinfo.st_size
264
os.stat in os.supports_dir_fd