Data sparsity is still the main challenge to apply machine learning models to solve complex scientific and engineering problems. The root cause is the “curse of dimensionality” in training these models. Training algorithms need to explore and exploit in a very high-dimensional parameter space to search the optimal parameters for complex models. In this study, a new scheme of multifidelity physics-constrained neural networks with minimax architecture is proposed to improve the data efficiency of training neural networks by incorporating physical knowledge as constraints and sampling data with various fidelities. In this new framework, fully connected neural networks with two levels of fidelities are combined to improve the prediction accuracy. The low-fidelity neural network is used to approximate the low-fidelity data, whereas the high-fidelity neural network is adopted to approximate the correlation function between the low-fidelity and high-fidelity data. To systematically search the optimal weights of various losses for reducing the training time, the Dual-Dimer algorithm is adopted to search high-order saddle points of the minimax optimization problem. The proposed framework is demonstrated with two-dimensional heat transfer, phase transition, and dendritic growth problems, which are fundamental in materials modeling. With the same set of training data, the prediction error of the multifidelity physics-constrained neural network with minimax architecture can be two orders of magnitude lower than that of the multifidelity neural network with minimax architecture.