I believe that is the starter solenoid. It would make a clicking noise because it is an electro-mechanical switch. A little magnet sort of thing flips on a contact switch in the solenoid when you push the start button. Power to the magnetic switch comes through the key switch, then then the start button; then power to the starter comes from the battery through the solenoid switch.
The reason it is set up this way is because there would be too big of a draw of electricity trying to go through the key switch/start button wires to the starter. The wiring in your key switch/start button would not last long trying to put all that amperage through and would soon cook the switches and/or wires. Notice how heavy the wires are on the solenoid? There is a lot of juice that runs through there and if you trace the heavy red wire, it should go from the battery then through the solenoid and on to the starter. Should be a smaller red wire that goes to the solenoid as well; that is the positive wire from the key switch/starter button. Normally, the ground connection for the solenoid is on the casing of the solenoid. If it is not connected to the frame in some way, (the usually used grounding source) then the solenoid won't work and it won't turn the starter. If it does not normally connect to the frame, then there would likely be a smaller black ground wire to the solenoid too.
I remember in the old days when the old beater car had a bad solenoid, I'd just touch a big screwdriver across the two heavy duty connectors on the starter solenoid. It had the same effect as if the switch IN the solenoid worked and it would turn the starter over. But boy oh boy would there be sparks from doing that!
So there you go. Not only did you learn what it is, but also how it works and why it is used.
HTH