Numpy mapping a 3D array onto a 2D array, but arrays don't match

Question

I'm not sure how the best way to go about this, but I have a data file with a list of coordinates x, y, and some magnitude. Lets say population.

X, Y, POP
1.2, 1.3, 1000
22.5, 2.5, 250
...
98.6, 1.7, 1500

So first, I round X, and Y to the nearest int and create a meshgrid based on a range of min and max values.

Xmin = np.amin(df['X'])
Xmax = np.amax(df['X'])

Ymin = np.amin(df['Y'])
Ymax = np.amax(df['Y'])

## I use a step of 10 as I don't have enough memory for a step counter of 1
## This is where the problem is
X = np.arange(int(Xmin), int(Xmax), 10) 
Y = np.arange(int(Ymin), int(Ymax), 10)

xx, yy = np.meshgrid(X, Y)

So now I have a meshgrid, which kind of contains all of my coordinates. The problem now lies in the fact, that I want this new mesh grid to contain my magnitude values from the dataframe.

Thus I want to map my df onto the meshgrid, to give me something like this.

X, Y, POP
1, 1, 1000
10, 1, N/A
20, 1, 250
...
90, 1, 1500

This is what I used to solve my problem, but it is slow, and compares every value. I suppose my question boils down to, is there a quicker/more efficient method compared to my code below? Ultimately, I plan to fill the N/A values using a mean of the surrounding cells. But to do that, I want to first map everything to a nice uniform grid.

state = np.empty([X.shape[0], Y.shape[0]])
state[:] = np.nan

for i in range(0, len(X), 1):
    for j in range(0, len(Y), 1):
        for z in range(0, len(df['X']), 1):
            if (abs(X[i] - df['X'][z]) < 10) and (abs(Y[j] - df['Y'][z]) < 10):
                state[i,j] = df['POP'][z]

You should consider doing a pandas join on the two arrays.

Onyambu
– Onyambu

2020-11-15 19:34:17 +00:00
Commented Nov 15, 2020 at 19:34 — Onyambu
– Onyambu, Commented Nov 15, 2020 at 19:34

Marat · Accepted Answer · 2020-11-15 19:01:54Z

0

precision = -1  # the number of digits to round off
df.groupby(
    [df['X'].round(precision), df['Y'].round(precision)]
).POP.sum().unstack().reindex(np.arange(...), np.arange(...))

I summed up values falling into the same cell, you might want to use average.

answered Nov 15, 2020 at 19:01

Marat

15.9k3 gold badges44 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Numpy mapping a 3D array onto a 2D array, but arrays don't match

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related