Update:
-New ladder system that saves you current position, you can move up and down the ladder, you can jump off the ladder too
-Lighting update
-Force Fullscreen

Bugs:
-Exiting the ladder can stutter the camera
-There's an issue with getting back onto the ladder after going up

Again the project is open sourced, you can use this in your project. When recreating, make sure player collision is always set to capsule instead of box because the ladder detection is buggy with box collider. And that the ladder entity is far enough away from the ground so the player doesn't clip through the ground when going down the ladder.

I'm working on a new first person movement system, right now this isn't compatible with that but I'll keep working on it.