The demonstration of nonlinear photonic functionalities is a prerequisite for the emergence of future optical computing schemes on chip. In order to develop these components, III-V semiconductors show many assets, from strong nonlinear coefficients to advanced technological processing possibilities. Recently, the emergence of III-V lasers grown on silicon has paved the way towards low-cost, large-scale integrated photonic devices [1] but this new platform is still mostly unexplored in the field of nonlinear photonics. Here, we report on the first demonstration of high quality-factor zinc-blende III-V microdisks on silicon. The unique GaP/Si photonic platform allows us to explore III-V crystal polarity engineering, a key concept in nonlinear photonics, from orientation-patterned to random quasi-phase matching. [2] We show that a random distribution of polarity domains can be achieved in monocrystalline GaP microdisks, with control over domain size and averaged polarity. The Q-factors measured, up to 16 000 in the telecom range [3] and 6300 in the visible range, allow us to assess for the first time the optical losses induced by polarity domain boundaries in these materials, a key parameter for the development of future polarity-engineered nonlinear photonic devices based of III-V semiconductors. The authors acknowledge the French National Research Agency for funding through the ANR project ORPHEUS (ANR-17-CE24-0019) and Nanorennes, member of the Renatech network.