You can use a file handle or you can work with file objects using with. Using with has the benefits of easier-to-read syntax and cleaner exception handling. Any files that are open will automatically be closed after your done when you use with.
Checking if a File Exists
from pathlib import Path
my_file = Path("/path/to/file")
if my_file.is_file():
# file exists
Looping Over File Object
myfile = "output.txt"
# file handle
file = open(myfile, "r") # a=append, r=read, w=write, rw=read/write
for line in file:
print(line)
# file object
with open(myfile) as file:
# read file line by line and output to list
data = file.readlines()
for line in data:
print(line)
Writing To a File
myfile = "output.txt"
# file handle
file = open(myfile, "w")
file.write("Write these lines to your file")
file.write("And close it when you're done")
file.close()
# file object
with open(myfile) as file:
file.write("Write these lines to your file")
file.write("And close it when you're done")
Writing to a File, Option 2
import sys
print("Send this error to stderr", file = sys.stderr)
f_handle = open('/tmp/output_20200511.txt', 'w')
print("Output to file", file = f_handle)
f_handle.close()
When trying to run make htmldocs from kernel source, it recommends doing so from a Python virtualenv. It’s a perfect time to document setting one up (per their recommendation).
$ which python3
/usr/local/bin/python3
$ which python
/usr/bin/python
$ which python3.6
/usr/local/bin/python3.6
You can create symlinks to safely reference items of interest:
$ ls -al /usr/local/bin/
total 24836
drwxr-xr-x. 2 root root 196 May 17 17:38 .
drwxr-xr-x. 12 root root 131 Apr 11 00:59 ..
-rwxr-xr-x. 1 root root 101 May 17 16:38 2to3-3.6
-rwxr-xr-x. 1 root root 242 May 17 16:38 easy_install-3.6
-rwxr-xr-x. 1 root root 99 May 17 16:38 idle3.6
lrwxrwxrwx. 1 root root 21 May 17 17:38 pip3 -> /usr/local/bin/pip3.6
-rwxr-xr-x. 1 root root 214 May 17 16:38 pip3.6
-rwxr-xr-x. 1 root root 84 May 17 16:38 pydoc3.6
lrwxrwxrwx. 1 root root 24 May 17 17:31 python3 -> /usr/local/bin/python3.6
-rwxr-xr-x. 2 root root 12699000 May 17 16:38 python3.6
-rwxr-xr-x. 2 root root 12699000 May 17 16:38 python3.6m
-rwxr-xr-x. 1 root root 3117 May 17 16:38 python3.6m-config
-rwxr-xr-x. 1 root root 441 May 17 16:38 pyvenv-3.6
Python 3 and pip
Issues like missing modules may arise. They’ll look something like the following:
$ python3 ./main.py
Traceback (most recent call last):
File "./main.py", line 8, in <module>
from lxml.etree import ET
ModuleNotFoundError: No module named 'lxml'
Can pip help us?
$ which pip3
/usr/local/bin/pip3
Looks like I have a pip3. Let’s try…
$ pip3 install lxml
-bash: /usr/local/bin/pip3: /usr/local/bin/python3.6: bad interpreter: No such file or directory
Hmmm…that does NOT look good. Do I have python3.6?
$ which python3.6
/usr/bin/python3.6
Sure do, but it’s in a location that pip3 is paying no attention to. As you can see from the output, pip3 is unusable. Just running pip3 results in bad interpreter: No such file or directory. We need to fix pip or fix the part where it’s expecting python3.6 to be in /usr/local/bin/python3.6
Easiest thing to try is to update the operating system’s hash table where it stores its mappings of programs to their locations so it doesn’t have to search every time. Sometimes those mappings can get out of sync, especially if a location of a program changes.
Maybe that will help pip3 find what it needs? Um, no. Not one bit.
$ pip3 install lxml
-bash: /usr/local/bin/pip3: /usr/local/bin/python3.6: bad interpreter: No such file or directory
Why is pip3 expecting python3.6 to be in that location when it clearly isn’t?
When symlinks are broken they blink.
After fixing my symlink (not because it would help with my current issue but because it’s good to fix things), I returned my focus to pip3. I would like to uninstall and reinstall it. If it is being managed by yum, it can easily be uninstalled by calling yum remove.
$ sudo yum remove python3-pip
Loaded plugins: fastestmirror
No Match for argument: python3-pip
No Packages marked for removal
But, it isn’t. How did pip3 get on this machine? Can python3 run pip as a module?
$ python3 -m pip
/usr/local/bin/python3: No module named pip
-m module-name
Searches sys.path for the named module and runs the corresponding .py file as a script.
Nope. Okay, enough spinning my wheels. The setup I have isn’t making much sense. How is this supposed to happen? How is pip supposed to be installed? Maybe I can do what’s supposed to be done (or by trying something else somebody else claims to have worked for them) and things will just work?
$ python3 -m ensurepip --user
Looking in links: /tmp/tmp41dea8fo
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages (39.2.0)
Collecting pip
Installing collected packages: pip
Successfully installed pip-18.1
$ python3 -m pip
Usage:
/usr/local/bin/python3 -m pip <command> [options]
Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
config Manage local and global configuration.
search Search PyPI for packages.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
help Show help for commands.
General Options:
-h, --help Show help.
--isolated Run pip in an isolated mode, ignoring environment variables and user configuration.
-v, --verbose Give more output. Option is additive, and can be used up to 3 times.
-V, --version Show version and exit.
-q, --quiet Give less output. Option is additive, and can be used up to 3 times (corresponding to
WARNING, ERROR, and CRITICAL logging levels).
--log <path> Path to a verbose appending log.
--proxy <proxy> Specify a proxy in the form [user:passwd@]proxy.server:port.
--retries <retries> Maximum number of retries each connection should attempt (default 5 times).
--timeout <sec> Set the socket timeout (default 15 seconds).
--exists-action <action> Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort).
--trusted-host <hostname> Mark this host as trusted, even though it does not have valid or any HTTPS.
--cert <path> Path to alternate CA bundle.
--client-cert <path> Path to SSL client certificate, a single file containing the private key and the
certificate in PEM format.
--cache-dir <dir> Store the cache data in <dir>.
--no-cache-dir Disable the cache.
--disable-pip-version-check
Don't periodically check PyPI to determine whether a new version of pip is available for
download. Implied with --no-index.
--no-color Suppress colored output
