Just a dumb question and obviously I don't even know why you want to access the server by its ip address, but if it's similar to my setup: why not setup a small dns server or dnsmasq?
From inside the lan the name resolves to the private address and from outside it resolves to the public address? I'm doing it like that having a very fast route to the server from inside the intranet.