There are two bolts retaining the starter to the engine block which can be accessed with standard wrenches. You would also need to remove the wires connected to the solenoid of which there are 2 or more often 3. (I have't looked at mine, but it's turning slow these days so it's going to be gone soon)
Technically, there is a solenoid, and a relay. This is FORD specific. While Chrysler and GM may have exactly the same, often the systems are only similar. (they have to do the same things, they just use different devices to do it)
The relay is on the fender, the solenoid is on the starter. The relay is responsible for taking a 12 volts on high gauge wires from the ignition switch to trigger a path for 12 volts connected to low gauge wires that can handle the current a starter requires (and are connected to the starter)
The solenoid is responsible to take the 12V on the lower gauge wires and energize a magnetic field that moves a shaft that forces the spur gear to engage with the flywheel , and also turn the starter motor. It takes a considerable amount of CURRENT at 12volts to turn the starter motor when it is trying to turn the engine.
Why tap on the solenoid?
Typcially tapping on a solenoid will allow the starter to function if the following failures occur:
The shaft that is supposed to move the spur gear out can develop corrosion that prevents it from moving. Typically in this failure mode you hear the starter spin, but the engine isn't turning over. The starter is just spinning. Tapping on it can free the shaft from this corrosion momentarily.
The contacts in the solenoid can fail such that an air gap is created between the internal contacts. Tapping on the solenoid can jar these contacts (typically a copper disk that can rotate) such that they move slightly, reducing the air gap sufficiently to let you start the car.
Both instances could happen again at any time, but this will often get you out of the parking lot of the video store you stopped at.