Creating new pandas dataframe in each loop iteration

Question

I have several pandas dataframes (A,B,C,D) and I want to merge each one of them individually with another dataframe (E).

I wanted to write a for loop that allows me to run the merge code for all of them and save each resulting dataframe with a different name, so for example something like:

tables = [A,B,C,D]

n=0
for df in tables:
    merged_n = df.merge(E, left_index = True, right_index = True)
    n=n+1

I can't find a way to get the different names for the new dataframes created in the loop. I have searched stackoverflow but people say this should never be done (but couldn't find an explanation why) or to use dictionaries, but having dataframes inside dictionaries is not as practical.

Why would putting dataframes in a dictionary not be practical? It is the straightforward way to do what you're suggesting. — Mark Whitfield
– Mark Whitfield, Commented Sep 20, 2017 at 15:18
It may be because I am not used to working with dictionaries really, and I don't really understand why creating the new variables in a loop is such a bad idea... — user5576
– user5576, Commented Sep 21, 2017 at 7:50

acushner · Accepted Answer · 2017-09-20 17:06:53Z

1

you want to clutter the namespace with automatically generated variable names? if so, don't do that. just use a dictionary.

if you really don't want to use a dictionary (really think about why you don't want to do this), you can just do it the slow-to-write, obvious way:

ea = E.merge(A)
eb = E.merge(B)
...

edit: if you really want to add vars to your namespace, which i don't recommend, you can do something like this:

l = locals()
for c in 'abcd':
    l[f'e{c}'] = E.merge(l[c.upper()])

edited Sep 20, 2017 at 17:06

answered Sep 20, 2017 at 16:00

acushner

9,9461 gold badge38 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user5576 Over a year ago

Could you please explain why it is not a good idea to automatically add variable names to the namespace? I see a lot of people saying that but never saw an explanation why...

acushner Over a year ago

@user5576 sure. it's because it's not explicit and clear (python.org/dev/peps/pep-0020). and it's usually a code smell. if you're programmatically creating a bunch of objects, you should operate on them in a programmatic way. instead of binding names n0..n25 to values, just put those values in a list and operate on them there.

Collectives™ on Stack Overflow

Creating new pandas dataframe in each loop iteration

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related