Convert a numpy array into a Android/Java readable format?

Question

I have a numpy matrix with dimension 7000*2048 with a type of int16, it takes around 40MB on my disk, I want to read this matrix in my android application, I tried converting it into a CSV file but the file size became more than 500MB which is too much to package in my android application.

I also tried converting it to JSON but again the file size becomes over 100MB, is there some suitable data format I can convert my numpy(int16) array into, that occupies less space and is readable in Android/Java.

what about custom binary format like: width, height, rest of data ...for 16 bit ints it will take (width*height + 2)*2 bytes ... remeber about endianness! — Selvin
– Selvin, Commented Jan 30, 2018 at 13:09
Why you don't use it in raw binary format and use FileInputStream? — Andrii Omelchenko
– Andrii Omelchenko, Commented Jan 30, 2018 at 13:11

A. Sam · Accepted Answer · 2018-01-31 13:46:14Z

For a simple array one can save the numpy data in a binary format, I first changed the data into a 2byte Signed Integer using data = data.astype(">i2") in python. As I am not saving any information on endianness , i am converting my numpy array to big endian. Then i can save the numpy values using numpy.ndarray.tofile() function.

Then i am reading this binary data in java using :-

   String file = "/home/sam/PycharmProjects/train.bin";
   FileInputStream fis = new FileInputStream(file);
   DataInputStream ds = new DataInputStream(fis);

   int count = ds.available();

   short features[][] = new short[count / (2 * 2048)][2048]; //i know the size of my data.
   int idx = 0;


   while (ds.available() > 0) {

    // read two bytes from data input, return short
    short k = ds.readShort();

    int row = idx / 2048;
    int column = idx % 2048;


    features[row][column] = k;
    idx++;

   }

Collectives™ on Stack Overflow

Convert a numpy array into a Android/Java readable format?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related