Due to the significance it holds in the concept of fraud, security in computers and business, anomaly detection serves very much purpose. Using techniques in unsupervised machine learning, the two algorithms applied in this study are Isolation Forest and Autoencoder in credit card fraud detection in financial datasets. This work focuses on data preparation and selection, generation and extraction of the features, as well as model assessment through use of metrics such as ROC-AUC, measure of precision, and measure of recall. It also discovered that Autoencoder attends to complex patterns of anomalies while Isolation Forest cuts down the false positives. Some of these problems include class imbalance and computational issues are highlighted. Some of these strategies include hybridization for imbalance handling and real time implementation that is very helpful in the development of automated and large scale anomaly detection in financial related work.