Complex-Valued Neural Networks: Learning Algorithms and Applications

Complex-valued data arise in various applications, such as radar and array signal processing, magnetic resonance imaging, communication systems, and processing data in the frequency domain. To deal with such data properly, neural networks are extended to the complex domain, referred to as complex-valued neural networks (CVNNs), allowing the network parameters to be complex numbers and the computations to follow the complex algebraic rules. Unlike the real-valued case, the nonlinear functions in the CVNNs do not have standard complex derivatives as the Cauchy-Riemann equations do not hold for them. Consequently, the traditional approach for deriving learning algorithms reformulates the problem in the real domain which is often tedious. In this work, we first develop a systematic and simpler approach using Wirtinger calculus to derive the learning algorithms in the CVNNs. It is shown that adopting three steps: (i) computing a pair of derivatives in the conjugate coordinate system, (ii) using coordinate transformation between real and conjugate coordinates, and (iii) organizing derivative computations through functional dependency graph greatly simplify the derivations.