There are a lot of uses for crew. Maintenance, analysis of data, taking care of passengers, guards, liaison officers, cargo haulers, I personally favor having a translator slave around.
However, when going into short missions in which I usually dock within the space of an hour, I don't need all these extra hands. I certainly don't need them to start interfering with manually loading ammo in the space of a 30 second frigate fight. It doesn't make any sense to take them aboard, especially since I'm usually fighting until the ship is destroyed. The odds of crew being killed in action on such an op is extremely high. Why would non-essential personnel go on such a mission?
I have standing maintenance crews etc. on my regular bases. I hire temps on stations I don't regularly visit when I need to to unload cargo and perform maintenance. Analysis and communication is done remotely.