This work presents a detailed, combined experimental and theoretical study on the structural stability of s-p bonded compounds with the BaAl4 structure type (space group I4/mmm, Z = 2) as part of a broad program to investigate the complex questions of structure formation and atomic arrangements in polar intermetallics. From ab initio calculations employing pseudopotentials and a plane wave basis set, we extracted optimized structural parameters, binding energies, and the electronic structure of the systems AeX(III)4, AeX(II)2X(IV)2, AeX(II)2X(III)2 (Ae = Ca, Sr, Ba; X(II) = Mg, Zn; X(III) = Al, Ga; X(IV) = Si, Ge). For all systems we found a pronounced pseudo-gap in the density of states separating network X42-bonding from antibonding electronic states that coincides with the Fermi level for an electron count of 14 electrons per formula unit, the optimum value for stable BaAI4-type polar intermetallics. However, the synthesis and structural characterization (from X-ray single crystal and powder diffraction data) of the new compounds AeZn2-δAI2+δ, AeZn2-δGa2+δ(Ae = Ca, Sr, Ba; δ = 0-0.2) and AeMg0.9Al3.1, AeMg1.7Ga2.3 (Ae = Sr, Ba) manifested that electron deficiency is rather frequent for BaAI4-type polar intermetallics. The site preference for different "X" elements in the ternary systems was quantified by calculating "coloring energies", which, for some systems, was strongly dependent on the size of the electropositive Ae component. The Ae2+ cations decisively influence the nearest neighbor distances in the encapsulating polyanionic networks X42- and the structures of these networks are surprisingly flexible to the size of the Ae component without changing the overall bonding picture. A monoclinically distorted variant of the BaAI4 structure occurs when the cations become too small for matching the size of encapsulating X42- cages. An even larger size mismatch leads to the formation of the Euln4 structure type.