I am trying to use the shape function for pandas to test if the dataframe has been chunked to 500000 rows each. But I can't due to my dataframe is now a pytable and when i use shape it gives me 'AttributeError: 'TableIterator' object has no attribute 'shape''.
Is there anyway I could convert from pytable to pandas dataframe?
Or is there a way to check if iteration is opened? As I want to see if pytable is pointing to the next object which means there are multiple chunks of 500000 rows returned.
The following is my code for testing:
def test_get_dataframe(workspace):
a = workspace.get_df('testing_df',True)
The get_df:
def get_df, df_name, chunk_mode=False, chunksize=500000):
return self._proj.get_dbHdf5().get_df(self._id, df_name, chunk_mode=chunk_mode, chunksize=chunksize)
"""
when chunk mode is true, return iterator to the chunks of the dataframe
"""
Aucun commentaire:
Enregistrer un commentaire