Whisk eggs, maple syrup, buttermilk, milk, vanilla extract, cayenne pepper, allspice, and 1/2 teaspoon salt together in a large bowl until smooth. Gradually mash sweet potatoes into egg mixture, starting with 2 tablespoons of sweet potato mixture, until all sweet potatoes are incorporated. Transfer sweet potatoes into prepared baking dish.