Fuel just keeps running and running

Just installed new fuel pump and new fuel injectors. I can hear the fuel constantly running through the system with just the key on. I have 42 psi at the rail, but shouldn't it pressurize the system then stop? The noise is back by the fuel pressure regulator. There are no leaking injectors.

Please help.

Fuel Pump

Pump should be running any time the ignition is on. If fuel pump relay is bad, pump could run even if ignition is off but that's another story. Fuel Pressure Regulator sends excess fuel back to the the tank through the return line. You might be confusing this with the ABS hydraulic pump which shuts off when pressure reaches the setpoint.