Populating a 2D array in python

Question

I need to populate a 2D array whose shape is 3xN, where N is initially unknown. The code looks as follows:

import numpy as np
import random

nruns = 5
all_data = [[]]
for run in range(nruns):
    n = random.randint(1,10)
    d1 = random.sample(range(0, 30), n)
    d2 = random.sample(range(0, 30), n)
    d3 = random.sample(range(0, 30), n)
    data_tmp = [d1, d2, d3]
    all_data = np.concatenate((all_data,data_tmp),axis=0)

This gives the following error:

ValueError                                Traceback (most recent call last)
<ipython-input-103-22af8f04e7c0> in <module>
     10     d3 = random.sample(range(0, 30), n)
     11     data_tmp = [d1, d2, d3]
---> 12     all_data = np.concatenate((all_data,data_tmp),axis=0)
     13 print(np.shape(data_tmp))

<__array_function__ internals> in concatenate(*args, **kwargs)

ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 0 and the array at index 1 has size 4

Is there a way to do this without pre-allocating all_data? Note that in my application, the data will not be random, but generated inside the loop.

Many thanks!

Do you understand what the error is saying the problem is?

Scott Hunter
– Scott Hunter

2020-12-30 00:38:40 +00:00
Commented Dec 30, 2020 at 0:38 — Scott Hunter
– Scott Hunter, Commented Dec 30, 2020 at 0:38

Tonechas · Accepted Answer · 2020-12-30 01:29:15Z

2

You could store the data generated in each step of the for loop into a list and create the array when you are done.

In [298]: import numpy as np
     ...: import random

In [299]: nruns = 5
     ...: all_data = []

In [300]: for run in range(nruns):
     ...:     n = random.randint(1,10)
     ...:     d1 = random.sample(range(0, 30), n)
     ...:     d2 = random.sample(range(0, 30), n)
     ...:     d3 = random.sample(range(0, 30), n)
     ...:     all_data.append([d1, d2, d3])

In [301]: all_data = np.hstack(all_data)

In [302]: all_data
Out[302]: 
array([[13, 28, 14, 15, 11,  0,  0, 19,  6, 28, 14, 18,  1, 15,  4, 20,
         9, 14, 15, 13, 27, 28, 25,  5,  7,  4, 10, 22, 12,  6, 23, 15,
         0, 20, 14,  5, 13],
       [10,  9, 23,  4, 25, 28, 17, 14,  3,  4,  5,  9,  7, 18, 23,  9,
        14, 15, 25, 26, 29, 12, 21,  0,  5,  6, 11, 27, 13, 26, 22, 14,
         6,  5,  7, 23,  0],
       [13,  0,  7, 14, 29, 26, 12, 16, 13,  3,  9,  6, 11,  2, 19, 17,
        28, 14, 25, 24,  3, 12, 22,  7, 23, 18,  5, 14,  0, 14, 15,  8,
         3,  2, 26, 21, 16]])

answered Dec 30, 2020 at 1:29

Tonechas

13.8k16 gold badges52 silver badges85 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

mg547 Over a year ago

Perfect, this is exactly what I was looking for! It is also easy generalizable to cases where I have more than 3 columns of data. Thanks!

liuyu · Accepted Answer · 2020-12-30 00:43:30Z

1

See if this is what you need, i.e. populate along axis 1 instead of 0.

import numpy as np
import random

nruns = 5
all_data = [[], [], []]
for run in range(nruns):
    n = random.randint(1,10)
    d1 = random.sample(range(0, 30), n)
    d2 = random.sample(range(0, 30), n)
    d3 = random.sample(range(0, 30), n)
    data_tmp = [d1, d2, d3]
    all_data = np.concatenate((all_data, data_tmp), axis=1)

answered Dec 30, 2020 at 0:43

liuyu

1,40412 silver badges27 bronze badges

1 Comment

Daniel Hao Over a year ago

It'll be great to add some explanations to point what went wrong in PO.

Quang Hoang · Accepted Answer · 2020-12-30 01:40:22Z

How about using np.random only:

nruns = 5

# set seed for repeatability, remove for randomness
np.random.seed(42)

# randomize the lengths for the runs
num_samples = np.random.randint(1,10, nruns)

# sampling with the total length
all_data = np.random.randint(0,30, (3, num_samples.sum()))
# or, if `range(0,30)` represents some population
# all_data = np.random.choice(range(0,30), (3,num_samples.sum()) )
print(all_data)

Output:

[[25 18 22 10 10 23 20  3  7 23  2 21 20  1 23 11 29  5  1 27 20  0 11 25
  21 28 11 24 16 26 26]
 [ 9 27 27 15 14 29 29 14 29 18 11 22 19 24  2  4 18  6 20  8  6 17  3 24
  27 13 17 25  8 25 20]
 [ 1 19 27 14 27  6 11 28  7 14  2 13 16  3 17  7  3  1 29  5 21  9  3 21
  28 17 25 11  1  9 29]]

Collectives™ on Stack Overflow

Populating a 2D array in python

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related