I wouldn't. You might then have a problem with water getting sucked into the motor. I had this problem with the stock sized hole so an even bigger one would probably be worse.
Don't cut it any bigger because you probably would have a problem with water like he said. I did what you did with the filter, but left the hole stock size.
Thats how I have mine too, with a big K&N Filtercharger that is suposed to be on a 3.8 LX T-bird. I had the filter on it first and then one day decided I wanted to take the air silencer out. So the very next time I took it to the car wash it sucked in water! I used to always get the underbody wash which is why I think it happend.