In a well-seasoned 10-inch cast iron skillet, melt butter over moderate heat until foam subsides and reduce heat to low. Sprinkle brown sugar evenly onto bottom of skillet and heat, undisturbed, 3 minutes (not all brown sugar will be melted). Remove skillet from heat. Cut enough rhubarb into 1-inch pieces to measure 3 cups and arrange in circular pattern, rounded sides down, in one layer over brown sugar.
Sift together flour, baking powder, baking soda and salt. In another bowl beat together butter and sugar until light and fluffy, then beat in vanilla. Add eggs one at a time, beating well after each addition. With mixer on low speed, add flour mixture alternately with buttermilk and milk, beginning and ending with flour and beating until just combined.
Spoon batter over rhubarb in skillet, spreading evenly and bake at 350 degrees until golden, about 45 minutes or until a toothpick comes out clean.
Run a thin knife around edge of skillet and invert a plate over skillet. Keeping plate and skillet firmly pressed together, invert cake onto plate(you may need help). Serve warm or at room temperature with whipped cream, vanilla ice cream or creme fraiche. Serves 8 to 10.