python(numpy): how to read specific columns from CSV file?

Question

I have a CSV data file, 100 columns * 100,000 lows and one header.

First, I want to make a list containing 1st, 3rd, and 5th to 100,000th columns data of original CSV data file.

In that case, I think I can use the script like below.

#Load data
xy = np.loadtxt('CSV data.csv', delimiter=',', skiprows=1)
x = xy[:,[1,3,5,6,7,8,9,10,11 .......,100000]]

But, as you know, it is not good method. It is difficult to type and it is not good for generalization.

First, I thought the below script could be used but, failed.

x = xy[:,[1,3,5:100000]]

How can I make a separate list using specific columns data, separated and continuous?

hpaulj · Accepted Answer · 2017-09-21 02:50:31Z

np.r_ is a convenience function (actually an object that takes []), that generates an array of indices:

In [76]: np.r_[1,3,5:100]
Out[76]: 
array([ 1,  3,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
       20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36,
       37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
       54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
       71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87,
       88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99])

This should be usable for both xy[:,np.r_[...]] and the usecols parameter.

In [78]: np.arange(300).reshape(3,100)[:,np.r_[1,3,5:100:10]]
Out[78]: 
array([[  1,   3,   5,  15,  25,  35,  45,  55,  65,  75,  85,  95],
       [101, 103, 105, 115, 125, 135, 145, 155, 165, 175, 185, 195],
       [201, 203, 205, 215, 225, 235, 245, 255, 265, 275, 285, 295]])

John Zwinck · Accepted Answer · 2017-09-21 02:06:14Z

2

Just use the usecols parameter in np.loadtxt().:

https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.loadtxt.html

answered Sep 21, 2017 at 2:06

John Zwinck

252k44 gold badges346 silver badges459 bronze badges

Comments

unutbu · Accepted Answer · 2017-09-21 02:53:26Z

1

Another option is to define x by removing columns from xy:

x = np.delete(xy, [0,2,4], axis=1)

answered Sep 21, 2017 at 2:53

unutbu

886k197 gold badges1.9k silver badges1.7k bronze badges

Collectives™ on Stack Overflow

python(numpy): how to read specific columns from CSV file?

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related