Carpenter ant signs include leaving a sawdust or frass which is essentially a by product of all their chewing. As explained in our CARPENTER ANT CONTROL article, they do a good job of chewing through wood but don't in fact eat any of it. Based on your description, it sounds like you have them in this fence and probably at this joint too.

I suggest you get some DRIONE DUST and CYPERMETHRIN. Fill one of our HAND DUSTERS with the Drione and have it alongside you when you take the fence apart so it's ready to use. I'm pretty sure you'll uncover a nest and the Drione will be ideal to use for the treatment. Once you complete the work at that joint, I suggest you dust each of all the other joints you have to insure there aren't any other satellite nests on the fence. The Drione will do a much better job than a liquid spray treatment.

Once the fence has been treated, finish the work by doing a foundation spraying using the Cypermethrin. This will insure straggler ants won't find their way into your home.