Place the rectangle with corners $(0,0),(8,0),(8,4),(0,4)$. Let the square have a vertex $A=(a,0)$ on the bottom edge, with its two neighboring vertices $B=(b,4)$ on the top edge and $D=(8,d)$ on the right edge.
- Since $AB=AD=5$: $(b-a)^2+16=25$ gives $(b-a)^2=9$; and $(8-a)^2+d^2=25$.
- Since the square's sides at $A$ are perpendicular: $(b-a)(8-a)+4d=0$. As in the figure, $B$ is to the left of $A$, so $b-a=-3$, which gives $4d=3(8-a)$.
- Substituting into $(8-a)^2+d^2=25$: $(8-a)^2\left[1+\frac9{16}\right]=25$, so $(8-a)^2=16$, giving $a=4$ (the root in $[0,8]$), hence $b=1$, $d=3$.
- So $A=(4,0), B=(1,4), D=(8,3)$, and the fourth vertex (opposite $A$) is $C=B+D-A=(5,7)$, which lies outside the rectangle.
- Edge $DC$ crosses the rectangle's top edge $y=4$: parametrizing $D+t(C-D)=(8-3t,3+4t)$, setting $3+4t=4$ gives $t=\frac14$, so the crossing point is $\left(\frac{29}4,4\right)$.
- The overlap region is the quadrilateral $A(4,0), D(8,3), \left(\frac{29}4,4\right), B(1,4)$. By the shoelace theorem, its area is $$\frac12\left|12+\frac{41}4+25-16\right|=\frac12\cdot\frac{125}4=\frac{125}8=\boxed{15\frac58}.$$
- Answer is (D).
