I too found this rather puzzling at first, and it made me wonder if there was something important I was missing. Now, knowing that there’s not from your comment above, I don’t mind sitting back and enjoying it. It is very evocative, and I do sense it getting at something it would be harder to express in a more direct narrative.
I feel, though, as if the “we” in the second stanza offers a false hope that a narrator is going to come in and shed some light on what’s happening in the first stanza, introduce some characters in some definite relation with it – and then, when this doesn’t happen in the second, then maybe it's all going to come together in the third. And then it doesn't. How would it be with “A day-hike” instead of “we day-hike”? I like that, with the judge being the first definite figure to emerge (not counting the title).