Fabric matters more than size. Structured cloth skims; thin jersey clings.
An empire waist sits high under the bust, so nothing fitted ever touches your middle.