logo
Splunk Team

how to create directories and sub directories efficiently and elegantly in python 2.7?

  • Last Update :
  • Techknowledgy :

So rather than checking isdir, you would simply catch the exception thrown if the leaf directory already exists:

def Test():
   main_dir = ["FolderA", "FolderB"]
common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]

for dir1 in main_dir:
   for dir2 in common_dir:
   try: os.makedirs(os.path.join(dir1, dir2))
except OSError: pass

Another more succinct way is to do the cartesian product instead of using two nested for-loops:

for dir1, dir2 in itertools.product(main_dir, common_dir):
   try: os.makedirs(os.path.join(dir1, dir2))
except OSError: pass

How about:

import os
from itertools
import starmap

def Test():
   main_dir = ["FolderA", "FolderB"]
common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]

map(os.makedirs, starmap(os.path.join, zip(main_dir, common_dir)))

And if we're worried about os.makedirs() throwing errors:

import os
from itertools
import starmap

def safe_makedirs( * args):
   try:
   return os.makedirs( * args)
except:
   pass # Ignore errors;
for example
if the paths already exist!

   def Test():
   main_dir = ["FolderA", "FolderB"]
common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]

map(safe_makedirs, starmap(os.path.join, zip(main_dir, common_dir)))

Suggestion : 2

3 days ago May 21, 2021  · It cannot make subdirectories. For making multiple directories, we can make use of the os.makedirs() method. The path is optional in this method. We can create subdirectories using the os.makedirs() method. To use these methods, we need to import OS module first. Then use them to create directories. Let’s create some directories using these ... , The os module has in-built os.mkdir () method to create a directory in the system. path: The location wherein the user wants the directory to be created. ,I am trying to create a bunch of directories and sub directories at a specific location in my PC. My process is something like this:, Using os.makedirs () os.makedirs () method in Python is used to create a directory recursively. That means while making leaf directory if any intermediate-level directory is missing, os.makedirs () method will create them all. For example, consider the following path: 


def Test(): main_dir = ["FolderA", "FolderB"] common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]
for dir1 in main_dir: if not os.path.isdir(dir1): for dir2 in common_dir: os.makedirs("%s/%s" % (dir1, dir2))

def Test(): main_dir = ["FolderA", "FolderB"] common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]
for dir1 in main_dir: for dir2 in common_dir: try: os.makedirs(os.path.join(dir1, dir2)) except OSError: pass
def Test(): main_dir = ["FolderA", "FolderB"] common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]
for dir1 in main_dir: if not os.path.isdir(dir1): for dir2 in common_dir: os.makedirs("%s/%s" % (dir1, dir2))
def Test(): main_dir = ["FolderA", "FolderB"] common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]
for dir1 in main_dir: for dir2 in common_dir: try: os.makedirs(os.path.join(dir1, dir2)) except OSError: pass
for dir1, dir2 in itertools.product(main_dir, common_dir): try: os.makedirs(os.path.join(dir1, dir2)) except OSError: pass
import os from itertools
import starmap def Test(): main_dir = ["FolderA", "FolderB"] common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"] map(os.makedirs, starmap(os.path.join, zip(main_dir, common_dir)))

Suggestion : 3

Another more succinct way is to do the cartesian product instead of using two nested for-loops:,Just read the directory and get the files in the directory, then loop through it to perform your action.,the linux command to move the files of subdirecties into one level up respectively,I think you can just keep the list as it is and iterate over it. You'd then produce a scatter plot for each sublist in the outer list, as the items from the sublist should share the same marker, color and legend label.

for dir1, dir2 in itertools.product(main_dir, common_dir):
   try: os.makedirs(os.path.join(dir1, dir2))
except OSError: pass
from itertools
import islice

def n_grams(a, n):
   z = (islice(a, i, None) for i in range(n))
return zip( * z)​
def filterStartStop(lst, start, stop):
   it = n_grams(lst, 3)
for i in it:
   if i[0: 2] == start:
   break
for i in it:
   if i[1: 3] == stop:
   break
yield i[1]

a = range(10)
print(list(filterStartStop(a, (1, 2), (7, 8)))) >>
   [3, 4, 5, 6]
import pickle
import csv

def save_bank(bank, f_stem):
   ""
"
Function to save out the bank
Pass in the bank and a file stem 'f_stem'
like 'bank1'
This
function will then save a 'bank1.csv'
and a 'bank1.pickle'
""
"
#Save out a csv of the bank so it 's human readable
with open(f_stem + ".csv", "wb") as out_csv:
   csv_out = csv.writer(out_csv, bank)
csv_out.writerow(['Name', 'password', 'money', 'misc']) #Write out a header
for account in bank:
   csv_out.writerow(account) #Write out each account as a row

#Save out a pickle of the bank so it 's python readable
pickle.dump(bank, open(f_stem + ".pickle", "wb"))

def load_bank(f_stem):
   ""
"
Function to load in the bank
Only needs the file stem used to save out the bank, i.e.
'bank1'
""
"
bank = pickle.load(open(f_stem + ".pickle", "rb"))
return bank

#Make a test bank
test_bank = [
   ["matt", "passw", 500, {
      "apple": 3
   }],
   ["luke", "passw123", 800, {
      "google": 2
   }]
]

f_stem = "bank1"
save_bank(test_bank, f_stem) #Save out the test_bank
new_bank = load_bank(f_stem) #Read it back in as a new bank

#See
if the read out and in are identical
print "Bank written and read correctly:", test_bank == new_bank

#Print out the new bank read back in
   print new_bank
>>> l1 = ['01/01/2005,01:00,0,0,0,2,0,0,2,0,0,2,0,0,2,0,0,2,0,0,2,0,0,2,0,3,E,9,3,E,9,8.0,A,7,6.0,A,7,87,A,7,1013,A,7,150,A,7,2.1,A,7,16100,A,7,77777,A,7,1.1,E,8,0.099,F,8,0.160,F,8,0,1,A,7'] >>>
   len(l1)
1

   >>>
   l2 = ['01/01/2005', '01:00', '0', '0', '0', '2', '0', '0', '2', '0', '0', '2', '0', '0', '2', '0', '0', '2', '0', '0', '2', '0', '0', '2', '0', '3', 'E', '9', '3', 'E', '9', '8.0', 'A', '7', '6.0', 'A', '7', '87', 'A', '7', '1013', 'A', '7', '150', 'A', '7', '2.1', 'A', '7', '16100', 'A', '7', '77777', 'A', '7', '1.1', 'E', '8', '0.099', 'F', '8', '0.160', 'F', '8', '0', '1', 'A', '7'] >>>
   len(l2)
68

r = csv.reader(f)
for row in r: # row in r, not row in f
data.append(row) # append row to data - > 2 d - array of items in rows
PS C: \ > .\Python37\ Scripts\ virtualenv.exe py3env
PS C: \ > .\py3env\ Scripts\ activate(py3env) PS C: \ > python
Python 3.7 .7(tags / v3 .7 .7: d7c567b08f, Mar 10 2020, 10: 41: 24)[MSC v .1900 64 bit(AMD64)] on win32
Type "help", "copyright", "credits"
or "license"
for more information.
r '(AANVRAGER:)\W*(.*?)\W*(___LOCATIE)'

aanvrager = re.compile(r '(AANVRAGER:)\W*(.*?)\W*(___LOCATIE)')
for line in LinesRAW:
   OUT.write(aanvrager.sub(r '\1\2\3', line))

Suggestion : 4

In Java, how do I efficiently and elegantly stream a tree node's descendants?,How to read a file into a vector elegantly and efficiently?,Sum array of arrays (matrix) vertically efficiently/elegantly,How to list all files inside a directory including sub-directories efficiently?

So rather than checking isdir, you would simply catch the exception thrown if the leaf directory already exists:

def Test():
   main_dir = ["FolderA", "FolderB"]
common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]

for dir1 in main_dir:
   for dir2 in common_dir:
   try: os.makedirs(os.path.join(dir1, dir2))
except OSError: pass

Another more succinct way is to do the cartesian product instead of using two nested for-loops:

for dir1, dir2 in itertools.product(main_dir, common_dir):
   try: os.makedirs(os.path.join(dir1, dir2))
except OSError: pass

How about:

import os
from itertools
import starmap

def Test():
   main_dir = ["FolderA", "FolderB"]
common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]

map(os.makedirs, starmap(os.path.join, zip(main_dir, common_dir)))

And if we're worried about os.makedirs() throwing errors:

import os
from itertools
import starmap

def safe_makedirs( * args):
   try:
   return os.makedirs( * args)
except:
   pass # Ignore errors;
for example
if the paths already exist!

   def Test():
   main_dir = ["FolderA", "FolderB"]
common_dir = ["SubFolder1", "SubFolder2", "SubFolder3"]

map(safe_makedirs, starmap(os.path.join, zip(main_dir, common_dir)))

Suggestion : 5

The mode parameter is passed to mkdir() for creating the leaf directory; see the mkdir() description for how it is interpreted. To set the file permission bits of any newly created parent directories you can set the umask before invoking makedirs(). The file permission bits of existing parent directories are not changed.,Changed in version 3.7: The mode argument no longer affects the file permission bits of newly created intermediate-level directories.,Remove (delete) the file path. This function is semantically identical to remove(); the unlink name is its traditional Unix name. Please see the documentation for remove() for further information.,In the next example, walking the tree bottom-up is essential: rmdir() doesn’t allow deleting a directory before the directory is empty:

for fd in range(fd_low, fd_high):
   try:
   os.close(fd)
except OSError:
   pass
if os.access("myfile", os.R_OK):
   with open("myfile") as fp:
   return fp.read()
return "some default data"
try:
fp = open("myfile")
except PermissionError:
   return "some default data"
else:
   with fp:
   return fp.read()
with os.scandir(path) as it:
   for entry in it:
   if not entry.name.startswith('.') and entry.is_file():
   print(entry.name)
>>>
import os
   >>>
   statinfo = os.stat('somefile.txt') >>>
   statinfo
os.stat_result(st_mode = 33188, st_ino = 7876932, st_dev = 234881026,
      st_nlink = 1, st_uid = 501, st_gid = 501, st_size = 264, st_atime = 1297230295,
      st_mtime = 1297230027, st_ctime = 1297230027) >>>
   statinfo.st_size
264
os.stat in os.supports_dir_fd

Suggestion : 6

For python 3.5 and above, you can use pathlib.Path.mkdir to create a nested directory.,Import class Path from pathlib library.,C Examples,This example is also similar to Example 2. Here mkpath() is used instead of makedirs().

For python 3.5 and above, you can use pathlib.Path.mkdir to create a nested directory.

from pathlib
import Path
Path("/root/dirA/dirB").mkdir(parents = True, exist_ok = True)

For python 3.2 and above, you can use os.makedirs.

import os

os.makedirs("/root/dirA/dirB")

Example 3: Using distutils.dir_util

import distutils.dir_util

distutils.dir_util.mkpath("/root/dirA/dirB")

Suggestion : 7

Back to our task of wanting to list all elements in a folder. We already know the path.,The simple command os.listdir lists all strings, i.e., only the path filenames. Here and in all other examples, I use type hinting for additional code documentation. This syntax became available from Python 3.5 onwards.,Done! We have resolved the problem in less than 10 lines. Since I planned to have filesurvey as a list of tuples, I can easily transfer the result into the panda data frame and analyze it there to compute the totals saved in folders, etc.,To transfer the file path, we must first combine the filename and path. I have often seen the following constructs in the wild, and even used them when starting out. For example:

Assuming that you wish to get a listing of a particular path accurately, we start by selecting a user directory on a Windows 10 system, which is basically a reproducible example:

path_dir: str = "C:\Users\sselt\Documents\blog_demo"

The variables assigned upon execution immediately cause an error:

SyntaxError: (unicode error)
'unicodeescape'
codec can 't decode bytes in position 2-3: truncated \UXXXXXXXX escape

Simply avoid the Windows separator and instead write the path using Linux separators only:

path_dir: str = "C:/Users/sselt/Documents/blog_demo"

Use raw strings with “r” as a prefix to indicate that special characters should not be evaluated.

path_dir: str = r "C:\Users\sselt\Documents\blog_demo"

The simple command os.listdir lists all strings, i.e., only the path filenames. Here and in all other examples, I use type hinting for additional code documentation. This syntax became available from Python 3.5 onwards.

import os

from typing
import List

path_dir: str = r "C:\Users\sselt\Documents\blog_demo"

content_dir: List[str] = os.listdir(path_dir)


Trending Technology


Most popular in python