Python List Files in a Directory Guide – listdir VS system("ls") Explained with Examples

When writing Python scripts and applications, it‘s common to need to work with files and directories on the file system. Python provides several ways to list the files in a directory, notably the os.listdir() function and running the "ls" shell command via os.system(). In this guide, we‘ll take an in-depth look at both approaches and compare them so you can choose the best tool for your needs.

Working with Files and Directories in Python

Before we dive into the specifics of listing directory contents, let‘s briefly review some key concepts around working with files and directories in Python.

In Python, a file is a named location on disk used to persistently store data. Files are organized into directories (also called folders), which can contain other files and nested subdirectories. Every file and directory has a path that specifies its unique location in the file system hierarchy.

Here are a few key terms to understand:

• Directory: a named collection of files and subdirectories, like a folder
• Current directory: the directory your Python script is located in or running from
• Path: the location of a file or directory, either absolute from the root directory or relative to the current directory
• Home directory: the default directory for a user, often abbreviated with ~

Python provides a built-in module called os that enables you to interact with the operating system, including the file system. You‘ll commonly use functions in os and its submodules for tasks like:

• Listing files and directories
• Creating, renaming and deleting files and directories
• Changing the current directory
• Accessing the current directory path
• Checking if a path exists and if it‘s a file or directory
• Joining and splitting path strings

To use the os module, first import it in your Python script or interactive shell:

import os

Then you can access its functions using dot notation, like:

os.listdir()
os.getcwd()
os.path.join()

Now let‘s look at how to use os to list files in a directory.

Listing Directory Contents with os.listdir()

The os module provides a convenient listdir() function for getting a list of the files and subdirectories inside a given directory. You call os.listdir() with a single argument: the path of the directory to list. It returns a list of strings, with each string being the name of a file or subdirectory in that directory.

Here‘s a simple example:

import os

path = "/home/user/projects"
dir_contents = os.listdir(path)

print(dir_contents)

This would print out something like:

[‘file1.py‘, ‘file2.txt‘, ‘subdir1‘, ‘subdir2‘]

If you omit the path argument, os.listdir() will use the current directory by default. So this code will list the contents of whatever directory the script is located in:

dir_contents = os.listdir()

The returned list contains both files and directory names. You can check if a given item is a file or directory using functions in os.path:

for item in dir_contents:
if os.path.isfile(item):
print(f"{item} is a file")
elif os.path.isdir(item):
print(f"{item} is a directory")

To get just the files or just the subdirectories, you can use a list comprehension to filter the initial list:

files = [f for f in os.listdir() if os.path.isfile(f)] subdirs = [d for d in os.listdir() if os.path.isdir(d)]

Here are a few more tips and things to keep in mind when using os.listdir():

• The list of names is returned in arbitrary order. It‘s not automatically sorted alphabetically or by any other criteria. Use the sorted() function if you need the results in a particular order.

• The . and .. special directory entries are not included. These refer to the current and parent directory respectively.

• If the path points to a file instead of a directory, os.listdir() will raise a NotADirectoryError exception. Be sure to handle this case if the path could be either a file or directory.

• On some operating systems, os.listdir() may return different capitalizations of names. For example, on Windows, the returned names will match the case of the actual names on disk, while on Unix-like systems the case is preserved as it was provided to listdir(). Use lower() or casefold() if you need to perform case-insensitive matching of names.

Running shell commands with os.system()

An alternative approach to listing directory contents in Python is to run the "ls" shell command via the os.system() function. This executes an arbitrary command in a subshell and returns the exit status.

To run "ls" and print the contents of the current directory:

import os

exit_status = os.system("ls")
print(f"Command exited with status {exit_status}")

A couple things to note here:

• The actual directory listing is printed to stdout by the "ls" command itself. os.system() just returns the numeric exit code, with 0 typically indicating success.

• This requires the "ls" command to exist on the system, which is standard on Unix-like operating systems but not on Windows by default. It will fail if run in a Windows environment without Unix command-line tools installed.

You can customize the behavior of "ls" by passing command-line flags. For example, to include hidden "dot files" in the listing:

os.system("ls -a")

To print one file per line with additional metadata about size, permissions, etc:

os.system("ls -l")

Consult the "man page" for the ls command to see all available options:

os.system("man ls")

The big advantage of os.system() is you have the full power of the shell available, so you can combine commands with pipes, redirect output to files, and so on. However, it‘s not as portable across operating systems, and it‘s harder to work with the results programmatically since they‘re just printed as raw text rather than returned as structured data.

To capture the actual listing text, you can redirect the command output to a file and then read the file contents in Python:

os.system("ls > dir_contents.txt")

with open("dir_contents.txt", "r") as f:
dir_listing = f.read()

print(dir_listing)

However, at this point it‘s usually better to just use os.listdir(), or look into more advanced solutions using the subprocess module if you really need to run shell commands.

When to use os.listdir() vs os.system()

In general, os.listdir() is the better choice for most use cases of listing directory contents in Python:

• It‘s more portable across different operating systems and doesn‘t depend on external shell commands.

• It returns the directory contents as a list of strings, which is easy to work with and manipulate in Python code.

• It‘s straightforward and Pythonic, and avoids the security risks of running arbitrary shell commands.

The main reasons to prefer os.system("ls") would be:

• You need some advanced behavior supported by "ls" flags that isn‘t easily replicated with os.listdir()

• You‘re working in an environment where only the shell command is available for some reason.

• You want to use shell wildcards, pipes, or other advanced command line features.

• You need to support legacy code that already uses os.system().

But for simple listing of file names and directories, os.listdir() should be your go-to tool.

Example: Counting files by extension

Let‘s conclude with a practical example that shows the power of os.listdir(). Say you have a directory full of source code files, and you want to count how many files of each type there are based on the file extension. Here‘s a short Python script to do that:

import os
from collections import defaultdict

extensions = defaultdict(int)

for filename in os.listdir():
ext = os.path.splitext(filename)[1].lower()
extensions[ext] += 1

print(f"File extensions in {os.getcwd()}:")
for ext, count in sorted(extensions.items()):
print(f"{ext}: {count}")

This will print out something like:

File extensions in /home/user/project:
.txt: 5
.py: 3
.md: 1
.csv: 1

The defaultdict makes it easy to count frequencies, and os.path.splitext() is handy for extracting just the file extension. The result is a succinct report of the directory contents generated with a few lines of portable Python.

Hopefully this in-depth look at os.listdir() and os.system("ls") helps you understand the options for listing files in Python and choose the right approach for your needs.