I know that the site has to go down for maintenance occasionally, but it seems like 4/5 times I've tried to access it in the last few days the session times out, I can't make a connection to the server etc. I have tried on multiple devices and never have any problem accessing other .onion sites. Is this happening with others too?
Yes. This happens to everyone, always.
Seriously mate, Agora is just as famous for it's downtime as it's popularity. It has weeks where it seems like nobody can get on at all, other times (like between 06:45 and 06:52 GMT) it works smooth as silk.