The people at Nguyen Shack run a great, small and secluded place with 5 bungalows and a few rooms with shared bathrooms.
The bungalows have small terraces with hammocks and private bath, all open air with fans.
They are definitely " off the beaten path" as described, but an easy 20 min from Ninh Binh train station.
The staff is all local as is the owner so you can support and interact with the locals easily.
Rent a motor bike, the bike ride is a bit long, and explore the Boat tours, temples and villages in the area.
It's a quick 3 stop, 2 hour train ride from Hanoi.