There are a ton of circuits available online for power and low battery indicators.
Here is one that i have ordered. It has both combined into 1 LED. This one is pretty cheap and comes with a potentiometer so it can be adjusted. I'm pretty sure Li-ion batteries work a little differently so the indicator needs to be different from other batteries. Correct me if i am wrong. I will test it out when it arrives and let you know how it is. At that point i can post some photos.