Symmetric key cryptography is one of the cornerstones of security in the modern era of electronic communication. The symmetric key algorithms, known as the ciphers, are to satisfy certain requirements in order to be considered secure, which are broadly classified as Classical Attack and Physical Attack. We show new results in context of both the classical and physical attacks to advance the state-of-the-art. In classical attack, we first show an issue related to a common modelling using Mixed Integer Linear Programming (MILP). We provide a new MILP modelling that overcomes this issue and explore heuristic options to reduce the solution time taken by the MILP solver. Our analysis shows that the solution time can be improved nearly ten folds by using a proper heuristic. Second, we show how Machine Learning (ML) can be used as a generic tool in the analysis of the symmetric key ciphers. In the process, we demonstrate how the existing security notions (that do not use ML) underestimate the vulnerability of the ciphers. To the best of our knowledge, this is the first generic application of ML in this field. In physical attack, we start with new mathematical results related to the Differential Fault Attack (DFA) from the point-of-view of the cipher designer. Next, we make use of these results to propose a cipher named DeFault, which has an in-built resistance against DFA. While all other methods to thwart DFA rely on some form of duplication, DeFault has an inherent protection against DFA that does not use duplication and hence is the first-of-its-kind. Third, we analyse the so-called Infective Countermeasure that is used as a duplication based DFA countermeasure in more depth. We construct new schemes, show weakness of an existing scheme, and propose a simple patch to another scheme to fix its weakness, among other results. Lastly, we propose a low-cost countermeasure to a newly proposed fault model, named Statistical Ineffective Fault analysis (SIFA). In contrast to the existing SIFA countermeasures that rely on ...