Creating Dynamic Dataframes Using Dictionary

In this tutorial, we will discuss about reading multiple files in to dataframes and append all the files to form a single dataframe.

Photo by Fré Sonneveld on Unsplash

To read files in a specific folder or location , we use the os package that helps us to set the working location to the target folder.

Once the package is imported, you can use the getcwd() method to get the current working directory.

You can change the working directory as per the requirement using chdir() method.

After the setting the working directory, we can read multiple files into dataframes, and we use dictionary to achieve this . This is the most easiest method for reading multiple files into dataframes.

If you observe, we have created a empty dictionary named files. The os.listdir() returns a list with all the files in the current working directory.

Once the read operation is completed, the dataframe is being stored as a value with key as the file name.

If you observe, we have placed file names in the key field of the dictionary to access the dataframe.

We would be to able to perform all kinds of dataframe operations when we access the dataframe using this approach. Let us go through one of the most common operation i.e. dataframe appending.

Let us append all the dataframes of the dictionary into a single dataframe. In order to create a new appended dataframe, we need to create another dataframe that can hold the structure of these dataframes.

The new dataframe created has the same structure as the files in the dictionary and it is empty.

Now we run the for loop again to append the dataframes one after another creating a single dataframe.

You can observe the index is repeating as 0,1,2 as there are 3 rows in each file. For a single continuous index, we need to set ignore_index to True.

In this tutorial, we have learned how to read multiple files at the same time in to dataframes, and access them as desired. Also, we learned how to append dataframes. Hope you enjoyed this tutorial, thanks for reading. Keep reading. Keep Learning.

Data Science and machine learning enthusiast