Python File Handling

About Python File

Python file handling allows you to read, write, and manipulate files on your computer. It provides a simple and efficient way to work with files, making it easy to store and retrieve data. In Python, you can use built-in functions and methods to perform various file operations, such as opening a file, reading its contents, writing data to a file, and closing the file when you're done.

We strongly recommend you to practice the Python file handling code of this tutorial in VS Code or any Python IDE to get better understanding of Python file handling. Our online Python editor has not been designed to handle file operations, so you may encounter issues when trying to execute file handling code in the online editor. For a seamless experience, please use a local Python environment or an IDE that supports file handling operations.

Ways of File Handling in Python

Python provides several built-in functions and methods for file handling. Here are some of the most commonly used ones:

open(): This function is used to open a file and returns a file object. You can specify the mode in which you want to open the file (e.g., 'r' for reading, 'w' for writing, 'a' for appending).
read(): This method is used to read the contents of a file. It can read the entire file or a specified number of characters. This is a method of the file object returned by the open() function.
write(): This method is used to write data to a file. You can write a string or a sequence of bytes to the file. This is a method of the file object returned by the open() function.
close(): This method is used to close a file after you are done working with it. It is a method of the file object returned by the open() function. It is important to close files to free up system resources.
with statement: This is a context manager that allows you to work with files without having to explicitly close them. It ensures that the file is properly closed after its suite finishes, even if an exception is raised.

File Handling Using `open()` Function

The open() function is used to open a file and returns a file object. You can specify the mode in which you want to open the file (e.g., 'r' for reading, 'w' for writing, 'a' for appending).

The open() function takes two main arguments: the file name (or path) and the mode. The mode can be one of the following:

'r': Read mode (default). Opens the file for reading. If the file does not exist, it raises a FileNotFoundError.
'w': Write mode. Opens the file for writing. If the file already exists, it truncates the file to zero length. If the file does not exist, it creates a new file.
'a': Append mode. Opens the file for writing. If the file already exists, it appends data to the end of the file. If the file does not exist, it creates a new file.
'b': Binary mode. This can be added to any of the above modes to open the file in binary mode (e.g., 'rb' for reading in binary mode).
'x': Exclusive creation mode. Opens the file for exclusive creation. If the file already exists, it raises a FileExistsError. If the file does not exist, it creates a new file.
't': Text mode (default). This can be added to any of the above modes to open the file in text mode (e.g., 'rt' for reading in text mode).
'+' : Update mode. This can be added to any of the above modes to allow reading and writing to the file (e.g., 'r+' for reading and writing).

Open & Read a File

To open and read a file, you can use the open() function in combination with the 'r' mode and read() method. This reads the entire content of the file. If the file does not exist, it raises a FileNotFoundError. Here is an example:


# Open a file for reading
file = open('example.txt', 'r')

# Read the contents of the file
content = file.read()
print(content)

# Close the file
file.close()

Open & Write to a File

To open and write to a file, you can use the open() function in combination with the 'w' mode write() method. This erases the existing content of the file if it already exists. If the file does not exist, it creates a new file. Here is an example:

# Open a file for writing
file = open('example.txt', 'w')
# Write some content to the file
file.write('Hello, this is a test file.\n')
file.write('This file is used for demonstrating Python file handling.\n')
# close the file
file.close()

Open & Append to a File

To open and append to a file, you can use the open() function in combination with the 'a' mode write() method. This adds new content to the end of the file without erasing the existing content. If the file does not exist, it creates a new file. Here is an example:

# Open a file for appending
file = open('example.txt', 'a')

# Append some content to the file
file.write('Hello, this is appending a file.\n')
file.write('This file is used for demonstrating Python file appending.\n')

# close the file
file.close()

Open a Binary File

To open a binary file, you can use the open() function with the 'b' mode. This is useful for reading or writing non-text files such as images, audio, video files, or any other binary data. Here is an example:

# Open a binary file for reading
file = open('image.jpeg', 'rb')

# Read the contents of the binary file
content = file.read()
print(content)

# Close the file
file.close()

Create a File

To create a file, you can use the open() function with the 'x' mode. This creates a new file and raises a FileExistsError if the file already exists. Here is an example:

# Create a new file
file = open('newfile.txt', 'x')

# Write some content to the new file
file.write('This is a new file created using Python.\n')
file.write('This file is used for demonstrating Python file creation.\n')

# close the file
file.close()

Using `with` Statement with `open()` Function (Recommended)

The with statement is a context manager that allows you to work with files without having to explicitly close them.

This is the best and safest way as it closes the file automatically after the block of code inside the with statement is executed, even if an error occurs. This makes it a safer and more efficient way to handle files in Python.

This is the recommended way to handle files in Python as it provides better error handling and cleaner code. Here is an example of using the with statement with the open() function:

Open & Read a File using with statement


# Open a file for reading using with statement
with open('example.txt', 'r') as file:

  content = file.read() # Read the contents of the file
  print(content)

In this example, the file is automatically closed after the block of code inside the with statement is executed, even if an error occurs. This makes it a safer and more efficient way to handle files in Python.

Handling Large Files (Memory Efficient)

Loading large file into memory using file.read() can lead to memory issues, especially if the file is larger than the available memory.

To handle large files in a memory-efficient way, you can read the file in smaller chunks or line by line instead of loading the entire file into memory at once. This allows you to process the file without running into memory issues. Here are some examples of how to handle large files in a memory-efficient way:

Read File Line by Line

# Read a large file line by line using with statement
with open('largefile.txt', 'r') as file:
    for line in file:
        print(line.strip()) # Process each line (e.g., print it)

In this example, the file is read line by line, which is more memory-efficient when working with large files. The strip() method is used to remove any leading or trailing whitespace from each line before processing it.

Read File in Chunks

# Read a large file in chunks using with statement
chunk_size = 1024 # Define the chunk size (e.g., 1024 bytes)
with open('largefile.txt', 'r') as file:
    while True:
        chunk = file.read(chunk_size) # Read a chunk of the file
        if not chunk: # If the chunk is empty, we have reached the end of the file
            break
        print(chunk) # Process the chunk (e.g., print it)

In this example, the file is read in chunks of a specified size (e.g., 1024 bytes). This is another memory-efficient way to handle large files, as it allows you to process the file in smaller parts without loading the entire file into memory at once.

File Streaming (Advanced Pattern)

File streaming is an advanced pattern for handling large files that allows you to read and process the file in a streaming manner. This is particularly useful when working with very large files that cannot fit into memory. In Python, you can use the io module to create a file stream and read the file in a streaming manner. Here is an example of using file streaming to read a large file:

Read File Line by Line using File Streaming

# Read a large file line by line using file streaming
import io

def read_large_file(file_path):
    with io.open(file_path, 'r', encoding='utf-8') as file:
        for line in file:
            yield line.strip() # Yield each line (e.g., print it)

# Example usage
for line in read_large_file('largefile.txt'):
    print(line)

In this example, the read_large_file() function is a generator that yields each line of the file one at a time. This allows you to process the file in a streaming manner without loading the entire file into memory at once. The yield statement is used to return each line of the file as it is read, allowing you to iterate over the lines of the file efficiently.

File Handling with Encoding

When working with files that contain non-ASCII characters, it is important to specify the correct encoding when opening the file. This ensures that the file is read and written correctly, especially when dealing with international characters or special symbols.

In Python, you can specify the encoding using the encoding parameter of the open() function. Here is an example of how to handle files with encoding:

Read File with Encoding

# Read a file with encoding using with statement
with open('EncodeData.txt', 'r', encoding='utf-8') as file:
    content = file.read() # Read the contents of the file
    print(content)

Using the encoding parameter, you can write to a file with the correct encoding as well.

Open Different Types of Files

You can open different types of files, such as text files, binary files, and CSV files. Here are some examples of opening different types of files:

Open CSV File

To open and read a CSV (Comma Separated Values) file, you can use the csv module in combination with the with statement. This allows you to read the contents of a CSV file in a structured way. Here is an example:

# Open a CSV file for reading using with statement
import csv

with open('data.csv', 'r') as file:
    reader = csv.reader(file) # Create a CSV reader object
    for row in reader:
        print(row)

In this example, the csv.reader() function is used to create a CSV reader object that allows you to read the contents of the CSV file in a structured way. The file is automatically closed after the block of code inside the with statement is executed, even if an error occurs.

Open JSON File

To open and read a JSON file, you can use the json module in combination with the with statement. This allows you to read the contents of a JSON file in a structured way. Here is an example:

# Open a JSON file for reading using with statement
import json

with open('data.json', 'r') as file:
    data = json.load(file) # Load the contents of the JSON file
    print(data)

In this example, the json.load() function is used to load the contents of the JSON file into a Python dictionary. The file is automatically closed after the block of code inside the with statement is executed, even if an error occurs.

Open Excel File

To open and read an Excel file, you can use the pandas library in combination with the with statement. This allows you to read the contents of an Excel file in a structured way. Here is an example:

# Open an Excel file for reading using with statement
import pandas as pd     # You nay need to install the pandas library using pip install pandas

with pd.ExcelFile('data.xlsx') as xls:
    df = pd.read_excel(xls, 'Sheet1') # Load the contents of the Excel file
    print(df)

In this example, the pd.ExcelFile() function is used to create an Excel file object that allows you to read the contents of the Excel file in a structured way. The file is automatically closed after the block of code inside the with statement is executed, even if an error occurs.

Open Word File

To open and read a Word file, you can use the python-docx library in combination with the with statement. This allows you to read the contents of a Word file in a structured way. Here is an example:

# Open a Word file for reading using with statement
from docx import Document   # you may need to install the python-docx library using pip install python-docx

with open('data.docx', 'rb') as file:
    doc = Document(file) # Load the contents of the Word file
    for paragraph in doc.paragraphs:
        print(paragraph.text)

In this example, the Document() function is used to create a Word document object that allows you to read the contents of the Word file in a structured way. The file is automatically closed after the block of code inside the with statement is executed, even if an error occurs.

Python Tutorial