Topological quantum computation with non-Abelian anyons offers a promising path toward fault-tolerant universal quantum computation. However, the practical realization of such a system remains challenging due to the difficulty of finding suitable topological materials. In this work, we provide a comprehensive blueprint for constructing a large-scale quantum computer based on the quantum double model $\mathcal{D}(S_3)$, a specific non-Abelian topological order. We implement logical computations using quantum circuits on qubits and qutrits, including a single non-Clifford gate, compatible with near-term quantum devices. This work bridges the gap between abstract mathematical frameworks and noise-resilient quantum computation on near-term devices. Our proposal offers a promising path to realize an anyon-based quantum computer.