The problem states to solve a recurrence relation, assuming that $T(k) = O(1) $ for $k \le 7$. The recurrence relation is $T(n) = 3T(\frac{n}{3}) + 5n$
Here is my work:
$T(n) = 3T(\frac{n}{3}) + 5n$
$= 3(3T(\frac{n}{9} + \frac{5n}{3}) + 5n$
$= 9T\frac{n}{9} + \frac{15n}{3} + 5n$
$= 9T\frac{n}{9} + 10n$
$= 27T\frac{n}{27} + \frac{15n}{3} + 10n$
$= 27\frac{n}{27} + 15n$
$.....$
$3^kT(\frac{n}{3^k}) + 5kn$
Solving for $k$:
$\frac{n}{3^k} = 1$
$log_3n = k$
Substituting back in:
$3^{log_3n}T(\frac{n}{3^{log_3n}}) + (5log_3n)(n)$
Truth be told, I'm not sure where to proceed from here. I'm working without any solutions so I don't even know if my work above is correct. I would greatly appreciate if someone could look it over and tell me if I'm on the right track. My instinct is that the solution is going to end up being either $O(nlogn)$ or $O(logn)$, but I can't base a proof on instinct.
Here we have $T(n) = aT\left(\frac nb\right) + f(n)$ where $a=b=3$ and $f(n)=5n\in\Theta(n)$. Let $c^\star=\log_b a=1$, then $f(n)\in \Theta\left(n^{c^\star}\right)$, so the master theorem yields $T(n)\in\Theta\left(n\log n\right)$, which is the result you obtained. (Note that the base of the logarithm does not matter since it only varies by a constant.)
Intuitively, the work to split/recombine a problem is comparable to that of the subproblems; at each level we have generate $3$ times as many problems $\frac13$ the size of the current problem, and the added work for each subproblem is proportional to the size of the current problem. So the recursion tree ends up having height $\log n$, with each level taking time $O(n)$. Hence, $T(n)\in O(n\log n)$.
Indeed, with the initial condition $T(1)=1$ we have $$ T(n) = n+5n\log_3 n,\quad n=3^k,\ k=0,1,2,\ldots $$ as can be proved by induction.