File Handling in Python

The primary principle that computers play by is input, processing and output. A computer takes some data, processes it, and outputs the result. The input and output may come in myriad of types of media, and one of the most used forms of input and output is the file.

We are so accustomed to using files that never stop to think that files are actually the best medium of input and output. As such, when beginners learn programming they often forget to hone their file handling skills. In real life scenarios we cannot progress from the beginning stage if we do not learn to handle files. If we cannot store the processed data of our input then what is the point of doing the processing? In python working with files is really fun and easy so I’ll try and make this article as easy, fun, and as effective as possible. I am going to discuss reading, writing, and listing files.

Let's start with a story of my own. Years ago I had a smartphone with Symbian OS, I installed a new version of an app that I really liked. The new version started to create some problems though. It was overwriting my registration of the previous installation without using that as it was supposed to.

At the moment I did not renew the new registration but I needed to use the app immediately. I was still a teen and not old enough then to get whatever and whenever I liked. So, I wanted to find a way out to freeze the app and stop its behavior. But it was not on a desktop computer where I could do whatever I liked, it was a small smartphone OS. So, to view the changes in the file system I uninstalled the app and then listed all the files in the system folders and app data folders with the help of Python. Then I installed the app again and listed the files once more and programmatically checked if any new files were there that I was not aware of. And yes there were two of them. Then I just made those two files read-only and started the app.

Now everything was alright. The app was not able to change the readonly files and it served my purpose successfully. I should tell you that it was the age of Python 2.2. As I am writing this article we have beta version of Python 3.7.

So, let's do what I did in a different way so that I can show you the most important aspects of handling files in Python. As a reminder I am going to use Python 3.x

Listing files

When we use Explorer on Windows, Finder on Mac and different file managers on linux distros we can easily see a list of files in various directories or drives with a few mouse clicks. We can do the same thing through the command line with ls on unix systems and dir in Windows. If this software was written in Python then how could Python do that? Yes, Python has some library functions for doing such operations. We have to import an OS module for our purpose. The OS module has a function in it called listdir(). Let's say we want to list the files in C:\testdata (Windows) or /srv/testdata (Unix/Linux). So, we run the following code.

import os
directory_path = r"C:\testdata"
files = os.listdir(directory_path)
for x in files:
   print(x)

The output on my machine appears as follows:

info1.txt
info3.txt
readme.md

Getting the size of a file

Now we have a list of files and we want to get the size of each file. To do that we can use a submodule of the OS called path. In the path submodule there is a function called getsize() that returns the size of a file. Remember that the size of a file is returned as byte count.

import os.path
file_path = r"C:\testdata\info1.txt"
file_size = os.path.getsize(file_path)
print("file size is: ", file_size, " bytes")

The output on my machine appears as follows:

file size is:  9  bytes

Reading files

A file is one of the most common ways computer programs get input data. Every time you open an application it gets some kind of input from files. Maybe your application has some kind of settings that needs to be persisted on every start of the application and in that case your program can consistently save the settings data in a file. When you are watching a movie or listening to music on your device, it is reading that video or audio data from files and outputting the result as video or audio. Now the question arises, how do we read files with Python?

Python has built-in functions called open() for opening a file and depending on the opening mode you can read or write files. The open() function takes several parameters for opening the file and returns a file object. The first and mandatory argument is the name of the file. If you want to open a file in the current working directory then you can just use the file name but if you want to open files from somewhere else on your file system then you have to provide the full path to the file. The second parameter is the opening mode of the file. It can be any of the following:

• Read mode: r
• Read as binary mode: rb
• Read and update mode: r+
• Read and update as binary mode: rb+
• Write mode: w
• Write as binary mode: wb
• Write and read mode: w+
• Write and read as binary mode: wb+
• Append mode: a
• Append as binary mode: ab
• Append and read mode: a+
• Append and read as binary mode: ab+

file_obj = open(r"C:\testdata\info1.txt", 'r')
text = file_obj.read()
print(text)
file_obj.close()

Output:

hEY mR Dj

After using the file object don't forget to close the file by calling the close() method on it. When we call read() without any (size) parameter then it reads the whole file into memory and that can lead to undesired situations e.g. crashing of the program, making the device unresponsive for a noticeable amount of time. So, we could specify how many characters (in text mode) or bytes (in binary mode) we want to read.

file_obj = open(r"C:\testdata\info1.txt", 'r')
text = file_obj.read(3)
print(text)
file_obj.close()

Output:

hEY

Even we can iterate over the lines in the file.

file_obj = open("a_text-file.txt", 'r')
for line in file_obj.readlines():
   print(line)
file_obj.close()

Writing to file

There is not much difference between writing and reading files. Set w as the file opening mode and in place of read() method use write() for writing to file. Let's say we want to write to a file in the current working directory named my-new-file.txt

file_obj = open("my-new-file.txt", 'w')
file_obj.write("This is line 1\n")
file_obj.write("This is line no 2\n")
file_obj.close()

Now open the file in a text editor/viewer and you will see that the same content is present there.

Appending to a file

By appending we mean writing to a file without overwriting the previous portion of data in the file. If you open a file in write mode and if the file name already exists in the file system then all the previous content is erased and ready for writing new data. If the file name does not exist in the file system then a new file with the name is created. But in append mode no previous content is erased. The write begins from the end of the file. Let's append to the previous file.

file_obj = open("my-new-file.txt", 'a')
file_obj.write("Hey, I am an appended line and I am supposed to be the third line in the fiel\n")
file_obj.close()

Look at the content of the file and you will see content the following:

This is line 1
This is line no 2
Hey, I am an appended line and I am supposed to be the third line in the file

The encoding parameter

There is an optional third parameter to the open() function that is not used frequently. It is a very important parameter when you are working with text files that is encoded in another encoding other than the default encoding. When you use unicode in your file do not forget to specify the parameter.

file_obj = open("my-new-file.txt", 'a', encoding='utf-8')

That's all for today. I am going to discuss more advanced topics regarding file operations in Python in another article in future.

*Stop by the homepage to search and compare SDKs, Dev Tools, and Libraries.