In classical neural networks, the challenges such as computational time remain because of the significant number of parameters. Quantum computing using quantum entanglement and quantum parallelism is an emerging computing paradigm that addresses this issue. Although quantum advantage is still studied in many research fields, quantum machine learning is a research area that leverages the strengths of quantum computing and machine learning. In this study, we investigated the quantum speedup with respect to the number of parameters in hybrid architecture. The proposed hybrid quantum residual networks (HQResNet) introduced the classical encoder and decoder to address the dimensionality mismatch problem in residual structures. The HQResNet outperformed classical and quantum benchmarks. Additionally, while previous studies adopted noise-free simulations, we conducted noise experiments considering NISQ devices, demonstrating that the hybrid model is robust to noise.