High resolution observations with ALMA have revealed that concentric rings and gaps are common features in protoplanetary disks. The favored mechanism for creating these substructures are planet-disk interactions, in which growing planets open gaps in the disk, and particles become trapped at the pressure maxima that form at the corresponding gap edges. Since the particle density in these pressure bumps can become much higher than the global value, they are likely sites for planetesimal formation via the streaming instability. We have studied the formation and fate of such planetesimals formed at planetary gap edges. By performing global simulations of an evolving disk perturbed by multiple planets, and including a state-of-the-art dust evolution model, we find that planetesimals indeed should form at planetary gap edges, and in significant amounts. Furthermore, the described process has a dramatic impact on the evolution of solids in protoplanetary disks, and therefore also on how the disks appear in observations. By using gravitational N-body simulations we demonstrate that the close proximity between the planetesimals and the planets causes the planetesimals to leave their birth locations soon after formation and spread out across the disk. In the current paradigm of planet-disk interactions, planetary gaps are often invoked as a mechanism to separate the disk into an inner and outer part that do not exchange material, but our results show that scattered planetesimals can in fact carry material past these gaps. Finally, we show that the accretion efficiency of these planetesimals onto the forming planets is very low. The high heavy element content of giant planets is often explained with planetesimal accretion during the gas accretion phase, but our results demonstrates that this is very unlikely. In conclusion, our work highlights that planetesimal formation at planetary gap edges can have huge implications for disk evolution and planet formation.