How to convert a Postgres bytea back into a float array?

Hi! I have a Postgres table I can read in MatLab via ODBC. The table has a column of type bytea. Each bytea value was created from a Python numpy array (a vector) of real (float64) values. I tried to use cell2mat() to get back the original array, but I ended up with a 1 x n array of uint8 values, not float64, and the element count didn't match my original vector, nor did any of the elements. Would anyone know how to convert a bytea value back to its original vector of float64 values within MatLab? Note - there could be ~100000 elements in the original vector.

------------------------------------------

That is, I'd like to follow this workflow, but with a lot more elements:

[1.1, 2.2, 3.3] -->

python numpay.array() -->

binary using Python pickle.dumps() -->

Bytea in Postgres -->

SOME METHOD IN MATLAB --> (this is the only part in question) -->

[1.1, 2.2, 3.3]

------------------------------------------

Thank you in advance!

Answer 1

[Rishav]

To convert the bytea value back to its original vector of float64 values within MATLAB, you need to follow the reverse process of how you created the bytea value in the first place. Here's an approach:

1. Read the bytea value from the Postgres table into MATLAB using the ODBC connection.

2. Convert the bytea value from a string to a uint8 array using the uint8 function:

bytea_str = % read bytea value from Postgres table
bytea_uint8 = uint8(bytea_str);

3. Decode the bytea value using Python's pickle.loads function. You can use MATLAB's py function to call Python functions:

% create a Python object for the bytea value
bytea_py = py.bytes(bytea_uint8);

% call pickle.loads function to decode the bytea value
numpy_array_py = py.pickle.loads(bytea_py);

4. Convert the resulting Python object (numpy_array_py) to a MATLAB array using the double function:

% convert Python object to MATLAB array
numpy_array_matlab = double(numpy_array_py);

5. Transpose the resulting array (numpy_array_matlab) to match the shape of the original numpy array:

% transpose the array to match the shape of the original numpy array
numpy_array_matlab = numpy_array_matlab';

After these steps, you should have the original vector of float64 values as a MATLAB array. Note that this approach assumes that the bytea value was created using Python's pickle.dumps function with a numpy array of float64 values.

However, if you used a different method to create the bytea value, you may need to adjust this approach accordingly.