Pandas Apply Function On Multiindex

I would like to apply a function on a multiindex dataframe (basically groupby describe dataframe) without using for loop to traverse level 0 index. Function I'd like to apply: def

Solution 1:

Using groupby on level with axis=1, let's you iterate and apply over the first level columns.

In [104]: (df.groupby("id").describe()
             .groupby(level=0, axis=1)
             .apply(lambda x: x[x.name].apply(CI, axis=1)))
Out[104]:
                                    a   b
id
0d1974107c6731989c762e96def73568  0.0 NaN
0fd4f3b4adf43682f08e693a905b7432  NaN NaN

Infact, you don't need CI, if you were to

In [105]: (df.groupby("id").describe()
             .groupby(level=0, axis=1).apply(lambda x: x[x.name]
             .apply(lambda x: 1.96*x['std']/np.sqrt(x['count']), axis=1)))
Out[105]:
                                    a   b
id
0d1974107c6731989c762e96def73568  0.0 NaN
0fd4f3b4adf43682f08e693a905b7432  NaN NaN

Sample df

In [106]: df
Out[106]:
           a         b                                id
47  0.218182       NaN  0d1974107c6731989c762e96def73568
48       NaN       NaN  0d1974107c6731989c762e96def73568
49  0.218182  0.130909  0d1974107c6731989c762e96def73568
50       NaN       NaN  0fd4f3b4adf43682f08e693a905b7432
51       NaN       NaN  0fd4f3b4adf43682f08e693a905b7432