How to Run a Shell Command in Python

Posted in Python by Dirk - last update: Feb 06, 2024

In Python, you can run shell commands using various methods. The most common way is to use the subprocess module, a versatile way to spawn new processes, connect to their input/output/error pipes, and obtain their return codes.

Subproces Module

The subprocess module in Python provides several functions for interacting with the operating system and executing shell commands.


import subprocess

command = "ls -l"
result = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

if result.returncode == 0:
    print("Command output:", result.stdout)
else:
    print("Error:", result.stderr)

In this example, the subprocess.run() function is used to execute the “ls -l” command, and the result is captured in the result variable. If something goest wrong - the error is captured and printed.

Here are the main parameters for the subprocess.run() function:

command (str or sequence of str): The command to be executed. If shell=True, this can be a single string containing the command and its arguments, or it can be a list of strings where the first element is the command, and the remaining elements are its arguments.
stdin (file-like object, optional): Standard input stream. By default, it is subprocess.PIPE. You can pass an open file object or subprocess.DEVNULL to indicate no input.
input (str, bytes, or None, optional): Input data to be sent to the process. If provided, it overrides stdin.
stdout (file-like object, optional): Standard output stream. By default, it is subprocess.PIPE. You can pass an open file object, subprocess.DEVNULL, or a file descriptor.
stderr (file-like object, optional): Standard error stream. By default, it is subprocess.PIPE. You can pass an open file object, subprocess.DEVNULL, or a file descriptor.
capture_output (bool, optional): If True, stdout and stderr will be captured and returned as part of the result object. Overrides stdout and stderr options.
shell (bool or str, optional): If True, the command will be executed through the shell. If a string, it specifies the shell to use.
cwd (str, optional): Current working directory for the subprocess.
timeout (float or None, optional): Maximum time the process is allowed to run, in seconds.
text (bool, optional): If True, the stdout and stderr streams are opened in text mode.

os.system Function

The os.system() function allows you to execute shell commands. It returns the exit code of the command.

import os

command = "ls -l"
exit_code = os.system(command)

if exit_code == 0:
    print("Command executed successfully")
else:
    print("Error executing command")

Unlike subprocess.run(), os.system() only returns the exit code of the command as an integer. It doesn’t provide detailed information about the execution, such as capturing the command’s output or error messages.

os.popen Function

The os.popen() function opens a pipe to or from a command. It allows you to read or write to the process.

import os

command = "ls -l"
output = os.popen(command).read()
print("Command output:", output)

os.popen() is convenient for simple cases, but it is less powerful than subprocess for more complex scenarios. subprocess offers more control over input, output, error handling, and other aspects of process execution. It returns a CompletedProcess object or a subprocess.Popen object, depending on the function used. This allows you to obtain information about the completed process, such as exit code, output, and more. os.popen returns a file object that represents a pipe connected to the process. You can read or write data to the process through this file object.

sh Library

The sh library provides a high-level interface for running shell commands. It aims to make it easy to run shell commands as if they were Python functions.

To use this library, you need to install it first:

pip install sh

import sh

result = sh.ls("-l")
print(result)

The sh library allows you to call shell commands as if they were Python functions. It aims to provide a more convenient and readable way to run shell commands, catering to users who prefer a simple and concise syntax.

On the other hand the subprocess module provides a lower-level, more versatile approach, suitable for a wide range of use cases, including scenarios where fine-grained control over process execution is required.

In summary, if you’re looking for a more Pythonic and convenient way to run shell commands with a simplified syntax, the sh module may be a good choice. However, if you need more control over the execution process, handling of input/output streams, and other advanced features, the subprocess module is a versatile and powerful option.

Reasons to run shell commands in Python

Running shell commands in Python is a common practice for various reasons:

Automation: Python can be used to automate repetitive tasks or workflows by executing shell commands. This is particularly useful for tasks such as file manipulation, data processing, or system administration.
System Interaction: Python scripts can interact with the underlying operating system to perform tasks that require system-level actions. This includes tasks like starting and stopping processes, managing files and directories, and configuring system settings.
External Tool Integration: many external tools and utilities provide command-line interfaces. By running shell commands in Python, you can easily integrate these tools into your scripts or applications, leveraging existing functionality without having to reimplement it.
Cross-Language Integration: Python can be used as a glue language to connect different components of a system. For example, a Python script might trigger a compilation process written in C or run a shell command to execute a Java program.
Data Science and Analysis: Data scientists and analysts often use shell commands to interact with data, perform data preprocessing, or execute command-line tools for statistical analysis. Python scripts can automate these tasks and provide a higher-level interface for data manipulation.
System Monitoring and Diagnostics: Python scripts can execute shell commands to gather system information, monitor resource usage, or perform diagnostics. This is valuable for system administrators or developers troubleshooting issues.
Deployment and Configuration: In deployment processes, Python scripts can be used to automate the setup and configuration of servers or deploy applications by running necessary shell commands. This is common in DevOps practices.
Parallel Processing: Python scripts can use multiprocessing to run shell commands concurrently, taking advantage of multiple CPU cores for improved performance in certain scenarios.
Platform Independence: While Python is platform-independent, some tasks may require platform-specific commands. Running shell commands allows you to perform platform-specific actions without needing to implement separate code for each platform.
Quick Prototyping: for quick prototyping or testing, running shell commands in Python provides a convenient way to experiment with system interactions before implementing more robust functionality.

Note: It’s essential to be cautious when running shell commands in Python, especially when dealing with user input or external data, to prevent security vulnerabilities such as command injection.