pickling objects imported with importlib.util

The easiest way to make it working is to manually add the module to the sys.module the following way:

import importlib.util
import sys

spec = importlib.util.spec_from_file_location('custom', 'C:\path\to\.py\file.py')
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
sys.modules['custom'] = module

In a similar application, I could make it work by doing something like:

file_path = 'C:\path\to\.py\file.py'
dir_name = os.path.dirname(file_path)
if dir_name not in sys.path:
   sys.path.append(dir_name)

Suggestion : 2

The purpose of the importlib package is two-fold. One is to provide the implementation of the import statement (and thus, by extension, the __import__() function) in Python source code. This provides an implementation of import which is portable to any Python interpreter. This also provides an implementation which is easier to comprehend than one implemented in a programming language other than Python.,Two, the components to implement import are exposed in this package, making it easier for users to create their own custom objects (known generically as an importer) to participate in the import process.,A concrete implementation of importlib.abc.SourceLoader by subclassing importlib.abc.FileLoader and providing some concrete implementations of other methods.,Import itself is implemented in Python code, making it possible to expose most of the import machinery through importlib. The following helps illustrate the various APIs that importlib exposes by providing an approximate implementation of importlib.import_module() (Python 3.4 and newer for the importlib usage, Python 3.6 and newer for other parts of the code).

try:
cache
except NameError:
   cache = {}

object
   +
   --Finder(deprecated) +
   --MetaPathFinder +
   --PathEntryFinder +
   --Loader +
   --ResourceLoader-- -- -- -- +
   + --InspectLoader |
   + --ExecutionLoader-- +
   + --FileLoader +
   --SourceLoader

suffixes = importlib.machinery.SOURCE_SUFFIXES
loader = importlib.machinery.SourceFileLoader
lazy_loader = importlib.util.LazyLoader.factory(loader)
finder = importlib.machinery.FileFinder(path, (lazy_loader, suffixes))

import importlib

itertools = importlib.import_module('itertools')

import importlib.util
import sys

# For illustrative purposes.
name = 'itertools'

if name in sys.modules:
   print(f "{name!r} already in sys.modules")
elif(spec: = importlib.util.find_spec(name)) is not None:
   # If you chose to perform the actual
import...
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
spec.loader.exec_module(module)
print(f "{name!r} has been imported")
else:
   print(f "can't find the {name!r} module")

import importlib.util
import sys

# For illustrative purposes.
import tokenize
file_path = tokenize.__file__
module_name = tokenize.__name__

spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)

Suggestion : 3

I saved a DataFrame to a file using pickle, then opened it using pickle in a different session:,combinatorial_data loaded as a DataFrame, despite pandas not being imported. Why is this - how can native python parse the data into pandas classes/objects? Or is pickle doing something fancy?

I saved a DataFrame to a file using pickle, then opened it using pickle in a different session:

import pickle
combinatorial_data = pickle.load(open("df.pkl", 'rb'))

Suggestion : 4

Exporters allow you to write packages of code, pickled Python data, and arbitrary binary and text resources into a self-contained package.,PackageImporter exposes complementary methods named load_pickle, load_text and load_binary that allow you to load Python objects, text and binary data from a package.,PackageExporter exposes three methods, save_pickle, save_text and save_binary that allow you to save Python objects, text, and binary data to a package.,Importers allow you to load code written to packages by PackageExporter. Code is loaded in a hermetic way, using files from the package rather than the normal python import system. This allows for the packaging of PyTorch model code and data so that it can be run on a server or used in the future for transfer learning.

$ unzip my_package.pt && tree my_package
my_package├──.data│├── 94304870911616. storage│├── 94304900784016. storage│├── extern_modules│└── version├── models│└── model_1.pkl└── torchvision└── models├── resnet.py└── utils.py
   ~cd my_package && cat torchvision / models / resnet.py
   ...

from zipfile
import ZipFile
with ZipFile("my_package.pt") as myzip:
   file_bytes = myzip.read("torchvision/models/resnet.py")
# edit file_bytes in some way
myzip.writestr("torchvision/models/resnet.py", new_file_bytes)

# add this to your .vimrc to treat `*.pt` files as zip files
au BufReadCmd *.pt call zip#Browse(expand("<amatch>"))

   ~ vi my_package.pt

with PackageExporter('my_package.pt') as pe:
   pe.save_pickle('models', 'model_1.pkl', mod)
# can limit printed items with include / exclude args
print(pe.file_structure(include = ["**/utils.py", "**/*.pkl"], exclude = "**/*.storages"))

importer = PackageImporter('my_package.pt')
print(importer.file_structure()) # will print out all files

# filtered with glob pattern:
   # include = ["**/utils.py", "**/*.pkl"], exclude = "**/*.storages"───
my_package.pt├── models│└── model_1.pkl└── torchvision└── models└── utils.py

# all files─── my_package.pt├──.data│├── 94304870911616. storage│├── 94304900784016. storage│├── extern_modules│└── version├── models│└── model_1.pkl└── torchvision└── models├── resnet.py└── utils.py

exporter_file_structure = exporter.file_structure()
found: bool = exporter_file_structure.has_file("package_a/subpackage.py")

Suggestion : 5

Last Updated : 15 Jul, 2022

Reloading modules in Python2.x

reload(module)

For above 2. x and <=Python3.3

import imp
imp.reload(module)

Reloading modules for >=Python3.4 and above

import importlib
importlib.reload(module)