Two-wheeled vehicles such as bicycles and motorcycles are widely used due to their compact dimensions, low inertia, and high maneuverability, making them suitable for navigation in confined spaces. However, maintaining dynamic balance, particularly at speeds below five meters per second, remains a significant challenge, as the stabilizing effects of gyroscopic precession and caster geometry diminish at low velocities. Addressing this issue requires advanced control strategies capable of handling highly nonlinear dynamics. In this study, the dynamic modeling of an autonomous self-balancing bicycle equipped with a reaction wheel was developed based on the Lagrangian formulation in MATLAB/Simulink. The state-space model was validated against a Simscape multibody simulation under identical pulse torque excitation, confirming the accuracy of the derived equations. The reaction wheel generates a counter-torque proportional to the roll angle deviation to restore the vehicle to its upright equilibrium. To enhance low-speed stability, two deep reinforcement learning (DRL) algorithms were implemented: the Deep Deterministic Policy Gradient (DDPG) and the Twin Delayed Deep Deterministic Policy Gradient (TD3). These model-free controllers optimize reaction wheel torque commands based on observed state variables without requiring an explicit dynamic model. A behavior-based navigation algorithm combining goal-reaching and obstacle-avoidance behaviors was integrated with the balance loop via a PID steering controller. The framework was benchmarked against a classical Linear Quadratic Regulator (LQR) across three scenarios: straight-path balance, curved-path navigation with four obstacles, and disturbance rejection under speed-bump excitation. Simulation results showed that the TD3 algorithm outperformed both DDPG and LQR, achieving a root mean square error (RMSE) of 1.23 degrees in roll angle during curved-path navigation. While the LQR controller performed adequately on straight paths, it exhibited significant oscillations during curved maneuvers due to nonlinear dynamics. The TD3 controller produced smoother torque profiles and superior disturbance rejection. These results confirm that DRL-based controllers offer a robust and adaptable solution for low-speed stabilization and autonomous navigation of self-balancing two-wheeled vehicles.