The most common indicator currently employed in practice to assess boiling water reactor (BWR) instability, due to density wave oscillation (DWO) is the Decay Ratio (DR), an easy to grasp index that is regularly calculated from an estimate of the impulse response function of the reactor core, such impulse response appraisal is most of the time provided by an autoregressive (AR) modeling of the reactor core. The DR is the output of most stability monitoring systems available in the market. However, it is known that BWRs are intricate systems that may exhibit complex dynamics during instability that cannot be captured by the DR alone. Besides, AR models require linear and stationary signals to grant reliable models. Recorded BWR signals are not linear and are not stationary. Therefore, it is necessary to reignite BWR stability studies to develop more suitable stability methodologies and indicators capable of accommodating the complex nature of unstable BWR signals. To address this issue, the work presented in this thesis is related to the study of non linear signal processing methodologies to assess the stability of a BWR, due to DWO. This thesis is divided in eight chapters. The first one introduces the various BWR instability types that have been observed in practice, it also introduces the DR definition. The second chapter introduces a state of the art of how BWR instability has been studied in the last three decades. The third chapter introduces the Empirical mode decomposition (EMD), a non-linear filter that accommodates non-stationary and non-linear behavior from real world signals. The EMD is the backbone of most of the BWR instability proposals given in this work. In the fourth chapter, a reduced order model (ROM) is studied, such ROM represents qualitatively the chaotic dynamic behavior of a BWR system during instability. The fifth chapter introduces the first non-linear instability indicator: The Shannon Entropy (SE) and its associated tests with BWR recordings is discussed in this chapter. The sixth chapter introduces the second non-linear instability indicator, the Sample Entropy (SampEn) and its experiments are discussed in this chapter too. Chapter seven discusses the third and final (and most powerful) non-linear instability indicator proposal: The Higuchi Fractal Dimension (HFD) and its associated experiments with artificial and real BWR signals are shown and discussed in this chapter. In Chapter eight, we introduce a novel and practical BWR instability monitor with decision rules based on the HFD for real time application. Final conclusions of the work performed in this thesis are given in chapter nine.