Pre-heat oven to 170C.
Roast the plums, sprinkled with brown sugar and a little sherry till they are soft; then leave to cool.
To make the custard base, place the milk, butter, sugar and lemon zest in a saucepan over a low heat, and stir till the butter and sugar are melted. Crumb the panettone in an electric blender and sprinkle the crumbs over the base of a 2 litre pudding dish. Add the whisked egg to the milk and pour over the crumbs. Leave until the crumbs have absorbed the custard.
Next bake the custard in a bain marie for 30-40 minutes till set.
Set oven to 150C.
Whisk the egg whites till stiff, add a spoonful of sugar and continue to whisk before adding the next spoonful and continue until all the sugar is incorporated.
Spoon the roasted plums over the baked custard, but do not add any of the plum juices. Next, spoon or pipe the meringue mixture over the plums and bake for 40-50 minutes.
I would serve this with cream.