core-dialog: fix bug where parent-dialog was used instead of nested dialog when checking top level dialog

Previously, when core-dialog attempted to find the top level dialog it only looked at the z-index of the various dialogs. However, this can cause an error when using nested dialogs. An example scenario could be (although maybe a bit contrived):
Open dialog A1, open nested dialog A2, open dialog B1, call dialog A1 again in order to have it appear above B1. A2 should be the top level dialog now.

core-dialog now considers the relation between dialogs now (if they are nested or not) and returns the correct top level dialog.