In attitude determination system (ADS) of a satellite, using artificial intelligence methods for estimating rotation quaternions based on sensor outputs, reduces computational costs while increasing the speed and accuracy. Additionally, it maintains its efficiency at sensor singularity points and, unlike traditional algebraic methods, enable attitude determination across the entire probable space of satellite positioning. hence, Attitude determination is a critical component in the design and operation of space systems. This study presents a novel framework for estimating the attitude of a geosynchronous satellite by combining Sun and Earth sensors with artificial intelligence algorithms. First, by modeling the satellite’s dynamic motion, unit vectors representing the positions of the Sun and Earth in both the orbital and body frames were generated, and a synthetic dataset was created by adding Gaussian noise to simulate sensor output. Using raw data led to inappropriate results in a way that in the best prediction, error of estimating rotation quaternion vector was more than 10 degrees, so to enhance the quality of the learning process, the raw data were augmented as a preprocessing step utilizing the TRIAD algebraic method. Several machine learning and deep learning algorithms, including XGBoost, CatBoost, and multilayer perceptron (MLP) networks, were then employed to estimate the satellite's rotation quaternions. The findings of this research demonstrated that the simultaneous and parallel use of four identical two-layer MLP models, each dedicated to predicting a single component of the quaternions from the preprocessed data, coupled with the normalization of the output quaternions from the neural networks, yields a mean absolute error of 0.285 degrees. This approach achieved approximately 10% performance superiority compared to the best method among those evaluated, namely the TRIAD algorithm, which makes it suitable for practical, real-time applications. This hybrid approach offers a cost-effective and reliable solution for implementing attitude determination systems in satellites.