Preheat oven to 350 F. Lightly grease a 9x13-inch baking dish with non-stick cooking spray.
Toss the torn Pumpkin swirl bread pieces into the bottom of the prepared baking dish. Add 12-15 dollops of the pumpkin butter evenly throughout the baking dish, so that each serving will have a little burst of pumpkin butter.
In a mixing bowl, whisk together the eggs, milk, pumpkin pie spice and vanilla. Pour over the bread pieces.
Cover with foil and bake in the preheated oven for 45 minutes, then remove the foil and bake for another 10 minutes.
Let cool slightly before serving warm. Add a little powdered sugar and cinnamon to a sifter and add a dusting over the top before serving. Other toppings ideas include butter, syrup, and whipped cream. Enjoy!