If I have a general uniform tesselation in hyperbolic plane (same configuration of regular polygons at every vertex, but multiple types of polygons allowed), how can I find the edge length and/or inner angles of the polygons so they'd fit?
For example: a truncated {3,7} tesselation has two hexagon and one heptagon at every vertex. What is its edge length and what are the inner angles of the two types of polygons?
On
The angle $\theta$ at a vertex of a regular hyperbolic $n$-gon with side length $s$ is given by
$$\sin\tfrac12\theta=\frac{\cos\tfrac12\tfrac{2\pi}n}{\cosh\tfrac12s}.$$
This is monotonically decreasing in $s$. As the polygon shrinks we have $\lim_{s\to0}\sin\tfrac12\theta=\cos\tfrac12\tfrac{2\pi}n$, or $\lim_{s\to0}\theta=\pi-\frac{2\pi}n$, the Euclidean $n$-gon angle. As the polygon grows we have $\lim_{s\to\infty}\theta=0$.
If your hyperbolic tiling has a vertex type $(n_1.n_2.n_3.\cdots)$, for the polygons to fit exactly around the vertex we need
$$2\pi=\sum_k\theta_k=\sum_k2\arcsin\frac{\cos\tfrac12\tfrac{2\pi}{n_k}}{\cosh\tfrac12s}.$$
This sum is also monotonically decreasing, so there is at most one solution $s$. There is a solution if and only if the sum in the Euclidean case $s\to0$ is greater than $2\pi$ (using the Intermediate Value Theorem). I don't think we can find the solution in closed form, in general.
Multiplying that equation by $\tfrac12i$ and exponentiating gives us this product of unit complex numbers:
$$-1=\prod_ke^{i\theta_k/2}=\prod_k\left(\sqrt{1-\left(\frac{\cos\tfrac12\tfrac{2\pi}{n_k}}{\cosh\tfrac12s}\right)^2}+i\frac{\cos\tfrac12\tfrac{2\pi}{n_k}}{\cosh\tfrac12s}\right);$$
since $\cos\tfrac\pi n$ is always an algebraic number, this shows that $\cosh\tfrac12s$ is also an algebraic number.
Start with the right triangle with corner angles
$$ \alpha=\frac\pi7,\quad\beta=\frac\pi3,\quad\gamma=\frac\pi2 $$
It is a building block for a tiling with regular heptagons, three meeting at each vertex.
The hyperbolic law of cosines gives you
\begin{align*} \cosh a &= \frac{\cos\alpha + \cos\beta\cos\gamma}{\sin\beta\sin\gamma} = \frac{\cos\alpha}{\sin\beta} \\ \cosh b &= \frac{\cos\beta + \cos\gamma\cos\alpha}{\sin\gamma\sin\alpha} = \frac{\cos\beta}{\sin\alpha} \\ \cosh c &= \frac{\cos\gamma + \cos\alpha\cos\beta}{\sin\alpha\sin\beta} = \frac{\cos\alpha\cos\beta}{\sin\alpha\sin\beta} = \cot\alpha\cot\beta \end{align*}
Now you compare this to the truncated heptagon. That's again a regular heptagon, so $\alpha$ and $\gamma$ stay the same, but you get a different $\beta'$, resulting in different lengths $a',b',c'$. The truncated heptagon is rotated inside the original one, so that its corners line up with the edge midpoints of the original. (I forgot this point in my first version of this answer.)
The distance between inner corner and outer edge is $b-c'$. If you want regular hexagons, then you have to make this equal to the length $a'$ of the shorter leg, since both of these are exactly half a hexagon edge. So you want to find a $\beta'$ such that
$$ b-c'= \operatorname{arcosh}\frac{\cos\beta}{\sin\alpha}-\operatorname{arcosh}(\cot\alpha\cot\beta') = \operatorname{arcosh}\frac{\cos\alpha}{\sin\beta'} = a' $$
I know of no closed formula to solve this kind of equation. I guess its transcendental, but I'm no expert there. You can solve this numerically, and will find something like
$$\beta'\approx1.0941107896648635317725955439550573051099051847044121941115690465$$
So instead of the $128.57°$ of a Euclidean heptagon or the $120°$ of the 7,3 tiling you get heptagons with interior angle $2\beta'\approx125.38°$. The interior angle of the hexagon is $180°-\beta'\approx117.31$ which of course compares to $120°$ in a Euclidean hexagon.
Using the $\beta'$ from above, you can find that the edge length of both the hexagons and the heptagons is
$$2a'\approx0.3343951728664474230865747047250440499075073624066706216059350025$$
The circumradius of the heptagon is
$$c'\approx0.3780772453203193756871600978881466991323285422408574087213795657$$