74LS138 and P-channel MOSFETS

I’m using an Atmega328, connected to a 74HC138 decoder for my 8×32 LED pixel (RGB) matrix. The outputs activate the 8 rows of the matrix though P-channel MOSFETS (Vgs 2-4 volts) which have12 volts on the source pin. When I use 5 volts, the matrix works properly, i.e. I can light individual pixels or LEDS. As I turn the voltage up (I need 12 volts because they are 1-watt LEDS), at about 8 volts other rows besides the two that I actually activate TURN on. At 12 volts all rows turn on, yet I’m only activating rows 1 and 2. I’ve tried an NPN transistor, an N-channel MOSFET to isolate the P-channel, various resistor changes, all to no avail. I intend to get rigorous today and instead of hacking, I'll actually take measurements of what works, what doesn’t work, but I’d appreciate any insight anyone could give. I've attached some relevant information.  I’ve since ordered some 74HC238 (sourcing o/p instead of sink) to try that with my N-channel-to-P-Channel driver! 

One more thing: I'm using Adafruit RGBMatrixPanel library and example codes.  In the code they use A0, A1 and A2 as digital output pins, so I do as well but I'm leery about using analog pins for DIO.

The  matrix is 5 feet wide as shown in photo.   The Logic board (MPU and 74138) is separate (small board at top of photo) from the Pixel boards that have the MOSFET drivers on them