On the Concorde, the APU did not exist. On both contemporary and modern planes, the APU is the Auxiliary Power Unit, which is a fancy way of saying electric generator. Basically a moderate-size internal combustion engine (relative to the plane; big planes have big APUs) with the usual arrangement to generate electricity.

Why? Well, it's handy if a plane can start its own engines, and big engines can't usually be started by batteries, so batteries start the APU and the APU starts the turbines. The APU is also used to drive climate and entertainment systems when on the ground (and not connected to the Ground Power Unit, also mentioned early in the thread).

Finally, the APU can provide fault tolerance: if the turbines are disabled in flight, the APU can provide hydraulic pressure for the control surfaces, as well as electricity for safety systems and the potential to restart the turbines. There is some correlation in turbine failures (flying through something, fire, one turbine blade flies out and destroys a second engine), but the APU is inside and less likely to fail at the same time.

Related is the Ram Air Turbine, which is basically a small wind generator that can pop out of the fuselage if needed to provide some power to critical system when really nothing is working.

Minor addition: engines need compressed air to start and APU provides that, too (used for engine starting, cabin pressurization and some anti-icing systems).