Why is there still power from my GPIO when in deep sleep?

I've got a fun project that I'm working on that is just about done until I came into this road block. Here is the idea of the project.

Wemos D1 Mini is in deep sleep

Button is press which connects reset and pulls board out from deep sleep

Dfplayer comes out of sleep

LED lights

Plays a tune

LED turns off

Dfplayer goes back to sleep

Wemos goes back into deep sleep

This is not a very complicated project and I thought I would have it knocked out pretty easy. The issue I'm having is that after the LED goes out and the board goes into deep sleep it relights but very dim. Currently the LED is plugged into D4 but I have had the same effect happen on other pins.

So bottom line, when the board goes into deep sleep the LED lights up dimly and I have no idea why?