This cave is called the Devil's Well and is located in the Bankhead forest. It got it's name from a 90 feet high waterfall that is inside the cave. Standing at the opening you can hear the rumble of the waterfall and the people who lived around there in the early 1900's told their children that was the sound of the devil and thus the name. Look closely at the first picture and see if you see anything "unusual" for lack of a better word or scary and not talking about being underground. I didn't see it either until much later after my visit. Hint: look in the bottom left ¼ of the photo.
Hiking through the Low Country I've seen areas like that quite a few times. Now, they aren't set out in a design like that one, but similar. They are old bunkers, of sorts, used in the Revolutionary and Civil War.