In rectangle $PQRS$, $PQ=10$ and $QR=6$. Points $A$ and $B$ lie on $\overline{PQ}$, points $C$ and $D$ lie on $\overline{QR}$, points $E$ and $F$ lie on $\overline{RS}$, and points $G$ and $H$ lie on $\overline{SP}$ so that $AP=BQ<5$ and $CQ=DR<3$. The convex octagon $ABCDEFGH$ is equilateral. Its side length can be written as $k+m\sqrt n$, where $k$, $m$, and $n$ are integers and $n$ is not divisible by the square of any prime. What is $k+m+n$?

1. Let $x$ be the side of the octagon, $PQ=a$, $QR=b$. As in the problem, $AP=\frac{a-x}2$ and $PH=\frac{b-x}2$, and $HA=x$ gives $\left(\frac{a-x}2\right)^2+\left(\frac{b-x}2\right)^2=x^2$, i.e. $x^2+(a+b)x-\frac{a^2+b^2}2=0$.
2. With $a=10,b=6$: $x^2+16x-68=0$, so $x=\dfrac{-16+\sqrt{256+272}}2=\dfrac{-16+4\sqrt{33}}2=-8+2\sqrt{33}$.
3. Check: $AP=\frac{10-x}2<5$ and $PH=\frac{6-x}2>0$ since $x\approx3.49$. So $k+m+n=-8+2+33=\boxed{27}$.