Beat egg yolks in a small bowl until smooth. Add about 3/4 cup of the hot milk mixture and stir it quickly into the egg yolks. Whisk the egg yolk mixture back into the hot milk mixture and continue to cook, stirring constantly, until thick and creamy, about 5 minutes. Turn off heat, add butter and vanilla, and stir until smooth. Let cool slightly, about 10 minutes.