np.diff() in NumPy

Recently, I was working on a data analysis project where I needed to analyze the rate of change between consecutive elements in a dataset. The issue is that calculating differences manually can be dragging and error-prone. This is where NumPy’s diff() function becomes invaluable.

In this article, I’ll cover how to use np.diff() effectively to calculate differences between array elements.

So let’s get in!

np.diff() in NumPy

In simple terms, np.diff() calculates the difference between consecutive elements in a NumPy array. It’s like asking, “How much did each value change from the previous one?” This function is incredibly useful for finding rates of change, detecting patterns, or identifying trends in your data.

Let’s start with the basics.

Read NumPy Divide Array by Scalar in Python

Basic Usage of np.diff()

First, you need to import NumPy:

import numpy as np

Here’s a simple example of using np.diff() with a 1D array:

# Create a sample array
temperatures = np.array([68, 71, 73, 75, 74, 77])
# Find the daily temperature changes
daily_changes = np.diff(temperatures)
print("Original temperatures:", temperatures)
print("Daily changes:", daily_changes)

Output:

Original temperatures: [68 71 73 75 74 77]
Daily changes: [3 2 2 -1 3]

Notice that the resulting array has one less element than the original. That’s because np.diff() calculates n differences from n+1 elements.

Check out np.unit8 in Python

Customize np.diff() with Parameters

Now, I will explain how to customize np.diff() with parameters.

1. Use the ‘n’ Parameter for Higher Order Differences

The ‘n’ parameter lets you compute differences multiple times:

stock_prices = np.array([145, 148, 153, 150, 157, 156])
# First order difference
first_diff = np.diff(stock_prices, n=1)
# Second order difference (difference of differences)
second_diff = np.diff(stock_prices, n=2)

print("Stock prices:", stock_prices)
print("First differences:", first_diff)
print("Second differences:", second_diff)

Output:

Stock prices: [145 148 153 150 157 156]
First differences: [3 5 -3 7 -1]
Second differences: [2 -8 10 -8]

I executed the above example code and added the screenshot below.

The second-order differences tell us about the acceleration or deceleration of the changes, which can be valuable for trend analysis.

Read NumPy Unique Function in Python

2. Work with the ‘axis’ Parameter for Multi-dimensional Arrays

For 2D arrays or matrices, you can specify which axis to calculate differences along:

# Monthly sales data for two products (rows) over 6 months (columns)
sales_data = np.array([
    [120, 135, 160, 155, 180, 190],  # Product A
    [95, 110, 105, 115, 130, 125]    # Product B
])

# Calculate month-to-month differences (along columns)
monthly_changes = np.diff(sales_data, axis=1)

# Calculate product-to-product differences (along rows)
product_diff = np.diff(sales_data, axis=0)

print("Original sales data:")
print(sales_data)
print("\nMonth-to-month changes for each product:")
print(monthly_changes)
print("\nDifference between products for each month:")
print(product_diff)

Output:

Original sales data:
[[120 135 160 155 180 190]
 [95 110 105 115 130 125]]

Month-to-month changes for each product:
[[15 25 -5 25 10]
 [15 -5 10 15 -5]]

Difference between products for each month:
[[-25 -25 -55 -40 -50 -65]]

I executed the above example code and added the screenshot below.

Using the axis parameter with np.diff() lets you control the direction of difference calculation in multi-dimensional arrays.

Check out Create a 2D NumPy Array in Python

Practical Applications of np.diff()

Let me explain to you some practical applications of np.diff().

1. Financial Time Series Analysis

np.diff() is commonly used in finance to calculate changes or returns between consecutive time points.

# S&P 500 closing prices for a week
sp500_prices = np.array([4535.38, 4500.21, 4515.55, 4550.58, 4585.45])

# Calculate daily returns
daily_returns = np.diff(sp500_prices) / sp500_prices[:-1] * 100

print("S&P 500 prices:", sp500_prices)
print("Daily percent changes:")
for day, change in enumerate(daily_returns, 1):
    print(f"Day {day} to {day+1}: {change:.2f}%")

Output:

S&P 500 prices: [4535.38 4500.21 4515.55 4550.58 4585.45]
Daily percent changes:
Day 1 to 2: -0.78%
Day 2 to 3: 0.34%
Day 3 to 4: 0.78%
Day 4 to 5: 0.77%

I executed the above example code and added the screenshot below.

This helps investors quickly analyze trends like daily gains or losses in stock prices.

Read NumPy Normalize 0 and 1 in Python

2. Scientific Signal Processing

In scientific signal processing, np.diff() is useful for analyzing how signals change over time.

# Simulating sensor readings with noise
import matplotlib.pyplot as plt

time = np.linspace(0, 5, 100)
signal = np.sin(2 * np.pi * time) + 0.2 * np.random.randn(100)

# Calculate rate of change
derivative = np.diff(signal) / np.diff(time)

# Plotting
plt.figure(figsize=(10, 6))
plt.subplot(2, 1, 1)
plt.plot(time, signal)
plt.title('Original Signal')
plt.subplot(2, 1, 2)
plt.plot(time[1:], derivative)
plt.title('Rate of Change (Derivative)')
plt.tight_layout()
plt.show()

By computing the rate of change, you can estimate the signal’s derivative and study its dynamic behavior.

Check out ValueError: setting an array element with a sequence error in Python

3. Image Processing – Edge Detection

In image processing, np.diff() helps detect edges by highlighting intensity changes across rows and columns.

# Simple gradient-based edge detection
import matplotlib.pyplot as plt
from matplotlib import image

# Load a grayscale image or convert to grayscale
img = np.array([
    [50, 50, 50, 150, 150, 150],
    [50, 50, 50, 150, 150, 150],
    [50, 50, 50, 150, 150, 150],
    [50, 50, 50, 150, 150, 150]
])

# Calculate horizontal and vertical gradients
horizontal_edges = np.diff(img, axis=1)
vertical_edges = np.diff(img, axis=0)

# Display
plt.figure(figsize=(12, 4))
plt.subplot(131)
plt.imshow(img, cmap='gray')
plt.title('Original')
plt.subplot(132)
plt.imshow(horizontal_edges, cmap='gray')
plt.title('Horizontal Edges')
plt.subplot(133)
plt.imshow(vertical_edges, cmap='gray')
plt.title('Vertical Edges')
plt.tight_layout()
plt.show()

This technique reveals horizontal and vertical edges, making it useful for basic gradient-based edge detection.

Read NumPy Average Filter in Python

Advanced Techniques with np.diff()

Let me show you some advanced techniques with np.diff().

1. Padded Differences to Maintain Array Size

Sometimes you want the output array to have the same dimensions as the input. Here’s a technique to pad the result:

def padded_diff(arr, padding_value=0):
    """Return differences with original array size by padding"""
    diff_result = np.diff(arr)
    # Add padding to maintain original size
    padded_result = np.concatenate(([padding_value], diff_result))
    return padded_result

# Monthly temperatures
monthly_temps = np.array([32, 35, 45, 58, 68, 78, 85, 82, 75, 62, 48, 35])
monthly_changes = padded_diff(monthly_temps)

for month, (temp, change) in enumerate(zip(monthly_temps, monthly_changes), 1):
    print(f"Month {month}: {temp}°F (Change: {change:+}°F)")

2. Combine with Other NumPy Functions

You can combine np.diff() with other NumPy functions like np.convolve() to smooth out short-term fluctuations and analyze trends more clearly.

# Sales data over 12 months
sales = np.array([15000, 16200, 18500, 17800, 19200, 25600, 
                  27800, 24500, 22300, 21500, 23400, 28900])

# Calculating moving average of differences
diff_3m_avg = np.convolve(np.diff(sales), np.ones(3)/3, mode='valid')

print("Monthly sales differences (smoothed):")
for i, avg_diff in enumerate(diff_3m_avg, 2):
    print(f"3-month avg ending month {i+1}: ${avg_diff:.2f}")

Check out np.abs() in Python Numpy

Common Mistakes and How to Avoid Them

1. Forgetting about dimension reduction: Remember, np.diff() returns an array with one less element along the differentiation axis. Always account for this in your code.
2. Using incorrect axis: When working with multi-dimensional data, selecting the wrong axis for differentiation can lead to unexpected results. Verify your axis choice with a small test case first.
3. Not handling NaN values: If your data contains NaN values, they’ll propagate through np.diff(). Consider using np.nansum() and manual calculation for those cases.

I hope you found this article helpful! The np.diff() function may seem simple, but it’s incredibly powerful for analyzing changes in data. Whether you’re working with financial data, scientific measurements, or image processing, understanding how to leverage this function effectively can significantly streamline your analysis workflow.

Other NumPy articles you may also like:

• NumPy Sum of Squares in Python
• Check if NumPy Array is Empty in Python
• np.round() Function in Python

Bijay Kumar

I am Bijay Kumar, a Microsoft MVP in SharePoint. Apart from SharePoint, I started working on Python, Machine learning, and artificial intelligence for the last 5 years. During this time I got expertise in various Python libraries also like Tkinter, Pandas, NumPy, Turtle, Django, Matplotlib, Tensorflow, Scipy, Scikit-Learn, etc… for various clients in the United States, Canada, the United Kingdom, Australia, New Zealand, etc. Check out my profile.

enjoysharepoint.com/