The long fill time assumes an empty tank, IIRC. None of us are going to run out tanks empty before filling. I'd only run mine down to about 2,500 psi before refilling it.
Buy a small compressor from somewhere like Northern. I bought a small Coleman there for less than 80 bucks for airbrush use. I put a water trap in the outlet line and will probably put a desiccant air drier after that. It's not a lot of cash outlay. It doesn't take much CFM to run it. You should be able to fill your rifle in short order with it, using a small airbrush type compressor.
I have a hand pump + 2 50cf tanks. I can use one while the other is filling. Fill the gun first, then use one tank while the other is being filled.