Using Requests, I need to send numpy arrays with json data in a single post to my flask application. How do I do this?
2 Answers
To convert a numpy array arr to json, it can be serialized while preserving dimension with json.dumps(arr.tolist()). Then on the api side, it can be parsed with np.array(json.loads(arr)).
However, when using the requests json parameter, the dumping and loading is handled for you. So arr.tolist() is all that is required on the client, and np.array(arr) on the api. Full example code below.
Client:
params = {'param0': 'param0', 'param1': 'param1'}
arr = np.random.rand(10, 10)
data = {'params': params, 'arr': arr.tolist()}
response = requests.post(url, json=data)
API:
@app.route('/test', methods=['POST'])
def test():
data = request.json
params = data['params']
arr = np.array(data['arr'])
print(params, arr.shape)
return "Success"
Output:
{'param0': 'param0', 'param1': 'param1'} (10, 10)
Note: When either the files or data parameter is being used in requests.post, the json parameter is disabled.
1 Comment
Julio Milani
This has incredibly low performance for large arrays. A much better way (at least 50% lighter) is to use tostring() and fromstring()
The accepted answer gets the job done for small arrays but has incredible low performance for large arrays (At least 150% of overhead).
I would recommend using tostring() instead of tolist().
EDIT 2023: .tostring() has been replaced with .tobytes() - both returns a copy of the raw data (much more efficient than sending a text representation of the array).
So client would become:
params = {'param0': 'param0', 'param1': 'param1'}
arr = np.random.rand(10, 10)
data = {'params': params, 'arr': arr.tostring()}
response = requests.post(url, json=data)
And API:
@app.route('/test', methods=['POST'])
def test():
data = request.json
params = data['params']
arr = np.fromstring(data['arr'],dtype=float).reshape(10,10)
print(params, arr.shape)
return "Success"
You should note that the shape and dtype of the array must be known beforehand or informed in the request body.
3 Comments
zjeffer
This doesn't work for me, because arr.tostring() returns bytes, which is not JSON serializable, so requests.post throws an error. What is the solution for this?
Julio Milani
You can also, encode it in base64: arr = base64.b64encode(np.array([1,2,3,4]).tobytes()).decode('utf-8') # arr = 'AQAAAAAAAAACAAAAAAAAAAMAAAAAAAAABAAAAAAAAAA=' vec = np.frombuffer(base64.b64decode(arr),dtype=int) # vec=[1,2,3,4]
Partha P. Das
encoding is a brilliant idea! my work is possible standing on the shoulders of giants.