jeudi 25 juin 2015

how to convert pytable to pandas dataframe

I am trying to use the shape function for pandas to test if the dataframe has been chunked to 500000 rows each. But I can't due to my dataframe is now a pytable and when i use shape it gives me 'AttributeError: 'TableIterator' object has no attribute 'shape''.

Is there anyway I could convert from pytable to pandas dataframe?

Or is there a way to check if iteration is opened? As I want to see if pytable is pointing to the next object which means there are multiple chunks of 500000 rows returned.

The following is my code for testing:

def test_get_dataframe(workspace):
a = workspace.get_df('testing_df',True)

The get_df:

  def get_df, df_name, chunk_mode=False, chunksize=500000):
    return self._proj.get_dbHdf5().get_df(self._id, df_name, chunk_mode=chunk_mode, chunksize=chunksize)

"""
when chunk mode is true, return iterator to the chunks of the dataframe
"""

Aucun commentaire:

Enregistrer un commentaire