Compared to urban areas, Type 2 diabetes has a higher prevalence and diabetes-related mortality rate in rural communities. Agentic Artificial Intelligence (AI) which refers to systems capable of autonomous reasoning, task planning, and adaptive behavior within a defined context can be a solution to this clinical issue. It utilizes a Large Language Model (LLM) that is created to understand human text and generate an understandable response. Using Retrieval-Augmented Generation (RAG), we can further enhance the capability of this framework by retrieving relevant data from a knowledge base to generate an understandable response. However, with current AI pipelines, it is challenging to evaluate every step that leads to an outcome. The objective of this project is to develop a preliminary agentic AI system that focuses on transparency when it comes to predictions, thereby increasing trust with the user and reducing knowledge-drift. In our research, we trained eight different models integrating machine learning (ML), SHapley Additive exPlanations (SHAP) for feature attribution, LLM variants, and RAG pipeline under varying conditions. While ML is providing good accuracy, we are exploring rule-based methods to adapt to the dynamic nature of underlying documents, and varying treatment guidelines thereby responding to patient needs.