This works summarizes the recent developments in GPT Models applicable to Financial Institutions. Comparative analysis highlights the strengths of proprietary models like BloombergGPT for domain-specific tasks and open-source alternatives such as FinGPT for flexibility and cost-effectiveness. The study also emphasizes the role of AI-powered platforms like AlphaSense in managing unstructured data for market intelligence. Proposed future work includes exploring alternative attention mechanisms, integrating multi-modal capabilities, and enhancing model interpretability to address the challenges of computational complexity and domain adaptation. The results demonstrate that GPT-based models significantly advance the capabilities of financial institutions to analyze large datasets, identify trends, and support data-driven decision-making. This paper contributes to the growing body of research on financial AI by proposing a scalable and effective approach to deploying LLMs in finance. This paper presents a comprehensive review of framework and architecture for leveraging Generative Pre-trained Transformers (GPT) in financial analysis and decision-making. Building upon the advancements in transformer-based models, the proposed approach integrates multi-layer self-attention mechanisms, fine-tuning on domain-specific data, and reinforcement learning with human feedback to enhance natural language understanding and generation tasks. Key contributions include an adapted GPT model with improved attention mechanisms and parameter scaling to handle financial texts effectively. The methodology involves a multi-phase approach encompassing data collection, pre-training, and fine-tuning, with performance evaluated using metrics such as perplexity, accuracy, and F1 score.