Authors : Venkata Ramana Reddy Bussu

Volume/Issue : Volume 9 - 2024, Issue 6 - June

---

Google Scholar : https://tinyurl.com/2ap5sbye

Scribd : https://tinyurl.com/3rb5mk6f

DOI : https://doi.org/10.38124/ijisrt/IJISRT24JUN417

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

---

Abstract : The integration of artificial intelligence (AI) with cloud-based analytics platforms has revolutionized data processing and decision-making. This research article explores the synergies between AI, Databricks, and Azure Data Lake Storage (ADLS), showcasing how organizations can harness AI capabilities to enhance data analytics workflows. Through a comprehensive analysis of real- world use cases, scalability assessments, performance optimizations, and cost efficiency evaluations, we demonstrate the transformative impact of AI-driven analytics on business outcomes.

Keywords : Azure Databricks, Unity catalog, Databricks Clusters, Spark, Data Intelligence, ML, Data Analysis, commerce, Data/AI, Azure Data Lakes storage.

References :

1. Goodfellow, I., et al. "Deep Learning." MIT Press, 2016.
2. Databricks: Unified Data Analytics Platform." Databricks, https://databricks.com/.
3. Azure Data Lake Storage: Scalable, Secure Data Lake Storage." Microsoft Azure, https://azure.microsoft.com/en-us/services/storage/data-lake-storage/.
4. Chollet, F. "Deep Learning with Python." Manning Publications, 2017.
5. TensorFlow: An Open Source Machine Learning Framework for Everyone." TensorFlow, https://www.tensorflow.org/.
6. PyTorch: An Open Source Deep Learning Platform." PyTorch, https://pytorch.org/.
7. Géron, A. "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow." O'Reilly Media, 2019.
8. Kumar, A., et al. "Scalable Data Processing with Apache Spark." IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 4, 2017, pp. 1013-1025.
9. Zaharia, M., et al. "Apache Spark: A Unified Analytics Engine for Big Data Processing." Communications of the ACM, vol. 59, no. 11, 2016, pp. 56-65.
10. Chiang, K., et al. "Azure Data Lake Storage Gen2: A deep dive into the service." Microsoft Azure Blog, https://techcommunity.microsoft.com/t5/azure-data-lake/azure-data-lake-storage-gen2-a-deep-dive-into-the-service/ba-p/267365.